Nigel,
I've done exactly that twice. I've built two "integrateds" using RCAs for multiple sources, a Grayhill selector switch, followed by a nice pot (the first was a 20K DACT, the second was the cermet pot that Hugh included with the GK-1... also 20K I think). This works extremely well.
I like the GK-1 and TLP-N better than this passive setup, but that is just a matter of taste. I think that many would find this passive setup extremely nice.
Peter