I had a similar setup for a while and simply had two pairs of speaker cables. I didn't switch that often, but it only took a couple of seconds to do it. It was easier to get to the back of the speakers than the back of the amps.
Some amps, such as Bryston, have two inputs (RCAs and XLRs). They are separated by a switch so that you could use two different pre sources, if you have, for example a preamp that uses XLRs, and a HT receiver that runs RCAs. Then you just have to flip the switch in the back.