I used to feel this way, and even went out & got an old Conrad Johnson tube amp that I could go direct into with a DAC / digital preamp as a parallel system to the HT one (i.e they would only share the main two speakers). But if you break it down, the "achilles heel" if you will of a HT + music system is the pre-amplification. Presumably the main speakers will be the same, and a decent amp (probably solid state) should be as good for HT as music, even if for HT it will be multichannel, so this will up the cost a bit.
It's really on the pre-amplification end where many HT receivers + processors are considered deficient compared to a good audiophile 2 channel preamp, probably because all of the potential additional low end circuitry the signal path could go through. Some people try to address this by getting very high end processors, others by getting preamps from Parasound, McCormak that are multichannel analog only (processing being done by the blu ray w/ multichannel analog out). Others might use the home theater bypass function on an integrated amp.
Alternatively I think there are two other options :
1) An amp that has dual inputs per channel - i.e. balanced & unbalanced - and use one for HT, and the other for music with a separate preamp or DAC / digital preamp. That why Parasound halo amps and some Anthem amps are attractive because they have both balanced and unbalanced. The Krell Showcase does as well, but I find the sound to be a bit too hard / brittle.
2) Recognize that some home theater receivers / processors actually are pretty decent preamps, particularly in the "pure direct" modes. No, they are not going to be as transparent as the very best expensive preamps, nor are they going to be as warm & euphonic as a tube preamp. But still damn good all the same if you listen objectively. I did a shoot out of my Marantz SR5005 & AV7005 vs an Anthem D1, which on the analog side is basically the same as the latest Anthem processors, and the Marantz's were not very far behind. The AV7005 was cleaner than the SR5005, and the D1 a bit warmer / meatier than the Marantz's, but not a hugh difference. So in summary I'm saying that the Marantz receivers / processors are really quite good even for 2 channel music, particularly if you use a good external DAC. The internal DAC's are ok, but not reference quality.
As for McCormak amps, I've only stayed away as my dealer claims their reliability was a bit iffy. He seemed to feel this way about Parasound amps as well because of his recent experience with customers ones melting down after minor mistakes (e.g. letting the spades on the positive / negative touch). He felt there should be better internal protections against these mistakes. Still would not keep me from buying a Parasound, but something useful to know that despite rugged looks, extra care should be taken in use.