If your "upgraded" parts measurably change the frequency response then the test is no longer valid. It needs to be controlled for any sort of validity. Otherwise we could just add an EQ and see which way we like it.
You need to read and view the measured data here on minor changes in frequency response.
http://www.audiocircle.com/index.php?topic=11132.0Minor changes in frequency response do NOT account for large scale changes in resolution, detail levels, imaging, noise floor, space between the notes, etc.
I think we definitely need a controlled double blind setup with a remote ABX switcher, some screen material to block the visibility of each speaker and ideally a rotating table which would place each speaker in the exact same position as the other when active.
First of all, you can't use 1 pair of speakers with two crossovers and a switch box between them unless the crossovers are first order.
Secondly, the speakers are the same speakers. They look alike! So no need to try to hide them.
Thirdly, you don't have to get real exact and technical if the differences are clear and identifiable in the first few seconds.
Here is how this has gone for me the past.
I set up an A/B comparison and have people all set to listen and write down the differences and what they hear and get everybody all set.... Then it becomes so night and day that no one wants to participate for more than 5 minutes. It gets to the point of it being like asking someone to pick which is Red and which is Blue. After one time through they look at you like you think they are stupid. You can't get people to do that for an hour.
Even having different speakers would make the test invalid.
Not true.
The music has to be very well known to the listener in order for it to be irrelevant. or there would have to be sounds or signals that would be neutral enough.
-The above comes from having gone to someone elses highly tuned system in an entirely new environment and trying to compare a dac. To a couple people familiar with that system, differences were immediately obvious that were difficult for me to pick up on. I think part of it was that I felt there was a bit of an expectaion for me to hear these distinctions, which was a distraction from the mental process required. After that experience I had occassion to compare a new cd player to a dac in my home and I had to go back and forth several times with a very small short piece of materials to try and tease out the nuances. But I was able to do it. Of course this kind of evaluation may be easier for other folks to do than myself. I do think there is some practice needed to learn how to retain a mental state to "listen for" things. This test may not really be as productive as it is hoped by some.
The speaker is the most critical component and I could imaine that the differences could be a lot more noticible than my experience in trying to differentiate one high quality digital source component from another, though.
There is a lot of truth to all of that. This is how double Blind ABX testing can be done to skew the results to one direction.
Case in point.
There was a get together one day over at Art's.
At one point I had everyone participate in some comparative listening. I had a DAC-60 set up in the system with a switch on the back that toggled between to sets of output coupling caps. One set was Axon poly caps, and the other was Sonicaps. Values were all matched.
Here is the wrench that always gets thrown into these things though, just like TRADERXFAN said:
Unfamiliar room, system, music, etc.
So we play a song all the way through, then switch and play it all the way through again. To no surprise, no one could say much one way or another. But a few people thought that they could hear some differences but would need to do it again to be sure.
That is the wrong way to conduct a comparative because it relies to much on memory retention.
So we confirm the obvious...
Then we really start to listen.
I play a small segment of a simple intro several times through. Everyone listens to it and learns how it sounds. Subtle details are memorized, etc. Then we switch. Right away differences are very apparent. EVERYBODY can tell. We go through it again and confirm. Not only is there a difference, but all agree that the difference is better in one verses the other. They discuss what they hear and agree on the differences.
We change to a difference piece of music and repeat with the same results.
Then we do it again with a different piece of music and now people are picking up on the signature of each one. One has a cleaner sound with more distinct space between the notes, less smearing, better resolution levels, etc. Everyone picks the two right away and most can pick which one it is without even hearing the other one.
Now that they have learned what to listen for you can't fool the group. I can play a short intro of anything and they can tell me which caps we are using without even hearing the other one, and in just a few seconds.
I got the same results at home with just me an other listener.
It is all in how you conduct the test.
Kevin,
If we can get this to happen I will be glad to send it up to you as well. We are going to have to do this with two sets of speakers and I need to use Dan's speakers, or his friends speakers as the second pair. So we will have to make sure he doesn't mind sending some as well.
