Let me preface the following comments by saying that I do not, in any way think that people who hear big differences in sound from tweaks are crazy.
I have only heard the sound change in my system twice when changing out components. The first time was switching from a $40 Sony progressive scan dvd player as my source to a Primare CD21 (current source). My system lost some upper treble glare and I was better able to discern the sound of specific instruments in each recording.
The second time is when I upgraded my amp from an Outlaw Audio RR2150 to my Odyssey Stratos Extreme SE. The stereo image became a little bigger and that was it. I've changed speaker cables three times and added a tubed preamp and have heard no differences. I ordered a NOS set of tubes to see if I can hear a difference in my preamp. I think some people are sensitive to subtle changes in sound so that the changes are no longer subtle. Others, like myself might be "tone deaf" in that we struggle to hear any, if at all difference. What I can tell you is that I am very happy with the sound coming out my stereo so in the end I believe everything has been worth it.
Just my thoughts, happy listening.
Regards,
Pete
P.S. I sometimes wonder if I put my Outlaw Audio RR2150 back in the loop (and took out my amp and preamp), what kind of changes I would hear. Alas, I no longer have the Outlaw so that test does not seem feasible at this point.