buy a four position dip switch. hook them up like this for each color:
red1------o dipswitch1
................../O---------line out
red2------o /
(dont mind the dots damn ubb wont let me leave spaces)
then just throw all four switches to switch the line.

just go to radioshack and ask for a 4 position dip switch they cost like $1
Mind you there are quite a few ways to do this, this is just the cheapest one.