[avahifi]

One of my engineers just built us a wonderful A-B switch box for us to use in evaluating prototype equipment.  It allows plugging in two preamplifiers,  or two DACs, and even two power amps and allow instant comparisons from a simply remote control.  It randomizes the connections to the two types of units under test to provide true blind testing.  It level matches of course to the nearest 0.2 dB. There are no switching transients at all, you can't hear when the switch from one unit to the other is made.

Our first test using this box was, of course, to test the box itself against a simple pass through straight wire connection.  The only part in the signal path inside the box, aside from some mechanical relays, is a digital controlled volume control.  This is essentially a chunk of silicon with a whole lot of resistive output connections.  There are no active devices in the signal path, no transistors, ICs, tubes or whatever.

The box passes the A-B against itself with flying colors.  When compared against itself with the digital volume pot in or out of circuit we can hear no change in the sound at all.

Next we have begun testing amplifiers.  Starting with the new Fet Valve 400R hybrid amp, we tested that against a new nearly ready for production solid state prototype amplifier.  The new solid state amp will replace the big Insight 440 and 260 series amplifiers and use the same regulated output circuit technology that has worked so well for us in the hybrid amps.  These new solid state super amps also include major audio circuit design upgrades and they are vastly better than our previous generation solid state amplifiers.  We actually can take orders for then now.  Call us at 651-330-9871 for details.

In normal listening evaluations here we were very happy to find that the new solid state amps improved even more than we had expected.  They are big, robust, dynamic, and smooth.  They image great and have no trace of their all solid state origins.  In our normal mode of listen to one amp for quite a while (hours) and then substitute another amp, we felt that the new solid state amps were a very close second to the Fet Valve hybrid amps.  They did not image quite as big, deep bass, although extraordinary, seemed not be be quite as powerful as the Fet Valve amps.  In other words, a very close second place, deserving of an untarnished Silver Medal, but still not the champion.  Inasmuch as they will sell for much less than the Fet Valve amps, we were really happy with the results.

Then we hooked up both amps to our new A-B comparison box.  We can't hear any difference between the new solid state prototype and the Fet Valve amp at all!  Both sound great, but we cannot tell them apart when we don't know which we are listening too.  Both are really good amplifiers, but these results kinda shocked me.

What is happening here?  Three possibilities:

1.  They actually are identical sounding even though both use wildly different technologies.

2.  The A-B switch box is a significant musical limitation, even though we thorough tested and listened to it in advance to make sure it is completely transparent.

3.  The actual A-B listening experience physoacousticaly modifies the way we are listening.  It takes us out of a just listening to great music mode into a "test" mode and that is causing us to mentally shut down.  I am really suspect that this is the case and really wish there were some documented evaluations about this issue. When I am forced to make an A-B comparison under test conditions, I suspect I am more involved into making sure I am providing the "right" answer than into just listening to the questions.

Using the A-B box, we certainly can hear the difference between a Fet Valve amp and a Ultravalve amp, but of course the ten times the power difference probably is an important factor here, especially when using inefficient speakers.

There certainly is a technical line of comments out there stating that all reasonably well designed power amps sound exactly the same in carefully done double blind AB test comparisons.  I simply don't accept these findings as the final word.  If I did, why would I be trying to design better amplifiers for the past 40 years?  I think something undocumented is going on when listeners are put into an always stressful A-B controlled listening test situation.  I think the test methodology itself affects the results.  How, I am not sure.  But when A-B testing of two different audio power amp designs makes them sound identical to me, then I would really like to be sure of what is actually going on.

I would appreciate any useful comments regarding this issue.

Thanks,

Frank Van Alstine

 

[Art_Chicago]

Very intersting, Frank, indeed.

Now--how much is the upgrade from say OmegaStar EX 260 to the latest??

[avahifi]

Hi Art,

No price on an upgrade yet.  It will be extensive.  New audio boards, each with six regulated power supplies built in, new ground plane boards, new output devices, and a ton of rewiring.  I will announce this when we have actually figured out the labor time to do it.

As of now, the price for the new Synergy 450 amp will be $1995 (black faceplate).  Other prices to be determined.

Frank

[charmerci]

My response to why you don't hear (small) differences in A/B switching between amps is best shown by a little trick. Press a coin hard against a forehead. It will stick for a while. Do this to someone telling them that. However, if you press the coin hard but also remove it, the person will still believe that it is stuck to their forehead.

So when you switch amps without the person knowing it, they still believe that they're still hearing the same amp.

[ajzepp]

Very interesting topic, Frank. I have always wanted to learn more about this, as well. The main problem I have had is that I simply don't wish to turn a hobby I love into a science experiment. I am very familiar with the objectivist crowd who believe in no qualitative differences among DACs, amps, cables, etc. I always tend to question my own experiences with gear, but I have neither the interest nor the time to create some properly designed A-B testing session to further my knowledge in this area. I guess in this case ignorance is bliss 

[Tom Alverson]

What happens when you A-B the Insight+ power amp and the new Synergy?  If they sound the same, does the A-B box lower the quality to the point they both sound the same?  Maybe and old fashioned non A-B test of the Synergy amp with the A-B box in line (several hours of listening, no switching back and forth) would tell you if it is changing the sound.

[Freo-1]

At first blush, it could be a combination of #2 and #3.

What is the input and output impedance of the switching box? 
Is the switching box using non-inductive resistors?   My tech buddy swears that resistors can and do modify the sound in some cases.  I did notice that after replacing the driver board resistors and the output emitter resistors on my Threshold 400A, there was a notable improvement in overall performance. It does stand to reason that any added components in the signal path have the ability to impact the sound.  For example, if one looks at the First Watt amps, they are purposely designed with as few parts as possible (but no fewer) to get the best possible sound within the design concept.    Tradeoffs are made with regard to measurements vs. design to get the desired sonic presentation.

I also think when you are listening to the two amps in this setup, it does impart a  psychoacoustic strain (less relaxed), looking for the answer. 

A couple of suggestions:
1)   As mentioned earlier, try listening without the box in the loop (matched up, of course).
2)   Try different (more demanding?) speakers.  Perhaps a different load will uncover additional information.

Thanks for sharing this with us Frank.  I’ll be keen to follow this thread and sees where it leads.

[Devil Doc]

I would think the Heisenberg Principle and the Observer Effect would explain your conundrum.

Doc

[Otis]

.

[kip_]

I would think the Heisenberg Principle and the Observer Effect would explain your conundrum.

If you're being serious... Quantum Effects only apply at Quantum distances and energy levels. Newtonian physics apply everywhere else.

[bunky]

I would think the Heisenberg Principle and the Observer Effect would explain your conundrum.

Doc

i was told by a good friend that if you A/B compare more than a few times in a listening session differences in sound can become obscured in the ear to brain interaction by what is refered to as habituation of stimuli. 

[trebejo]

I'm only thoroughly persuaded by qualitative differences when I do this and of course I'm nowhere near as able to have this fantastic a/b gadget.

Have you listened for speech intelligibility? Maybe a movie with the audio downsampled to mp3 (not that we would know how to obtain such a thing   ). That's not going to test the bass spectrum, but maybe some kind of qualititative test can be found there too.

The big problem then is that the holistic test is darn near impossible to divorce from psychoacoustics. I know I would want the solid state to sound better, so I wouldn't have to worry about changing the tubes anymore!

[*Scotty*]

Frank, if the two amplifiers imaged differently when you initially compared them and now they are imaging the same, then something has leveled the playing field. If one amplifier has noticeably better image height than the other and now they have the same height or width there is something funny going on.
 It's kind of hard to get this wrong especially if one amp will put an image on the ceiling and the other won't or if one amp has a much wider sound-stage than the other and consistently places images far outside of the boundaries of speakers. 
 As far as your A-B testing goes you might try an A-B-A test sequence. You listen to A first and get accustomed to it and then switch to B, then when you switch back to A again you may go A-HA, now I can hear what was added or subtracted by B. If there is a meaningful difference between the two amps it should be easy to hear.
 Another possibility is masking. Even though the switch box appears to be transparent and no different than a straight wire there could be enough masking present to obscure smaller differences and still allow larger magnitude differences to be perceived. I am wondering if the A-B box being in the system reduces the magnitude of the differences between the Fet Valve amp and the Ultravalve amp. You can still hear the difference between the two but perhaps they are not as far apart in performance as they were when they are compared against one another without the switch box in the system.
Scotty
 

[WGH]

I think something undocumented is going on when listeners are put into an always stressful A-B controlled listening test situation.

The obvious solution would be to make the test stress free. I remember back in the '70's we used to get very relaxed while listening to music but at the same time our hearing acuity was heightened, almost an out of body experience. Unfortunately I have no idea how to duplicate this state in 2012.

As the designer you will always have your analytical hat on while A-B testing. Test subjects shouldn't care a whit about the equipment or even what is being tested. The question to the test subject would be "Do you like it this way? Or this way?" If the music sounds the same to the test group then the A-B box would be masking the differences (or there are none). Wives would be a good starting test group.

Wayne

[rcag_ils]

What is happening here?  Three possibilities:

1.  They actually are identical sounding even though both use wildly different technologies.

2.  The A-B switch box is a significant musical limitation, even though we thorough tested and listened to it in advance to make sure it is completely transparent.

3.  The actual A-B listening experience physoacousticaly modifies the way we are listening.  It takes us out of a just listening to great music mode into a "test" mode and that is causing us to mentally shut down.  I am really suspect that this is the case and really wish there were some documented evaluations about this issue. When I am forced to make an A-B comparison under test conditions, I suspect I am more involved into making sure I am providing the "right" answer than into just listening to the questions.

Using the A-B box, we certainly can hear the difference between a Fet Valve amp and a Ultravalve amp, but of course the ten times the power difference probably is an important factor here, especially when using inefficient speakers.

Frank, you can pretty much rule out 2 and 3. Assuming the A/B box is transparent to musical signal which I think it is, if you could hear the differences between the Ultra and the Fetvalve, and hear no difference between the Fetvalve and the new SS amps, that tells me the A/B box is doing what it's supposed to do. I don't buy #3 at all because you did not mentally shut down when you compared the Ultra and the Fetvalve, and hear their differences. I don't see any possibility other than #1.

[Elizabeth]

The human mind is a sophisticated GROUP of parts which work together.
Sometimes one task can block others when attending to it.
So listening when relaxed and just enjoying the  music, a lot of different parts of the mind are involved.
Those are very different than the parts used to make decisions about a differentiation in the sound.. (Vs just listening for pleasure)

A post mentioned the principle of you can't know where a partical is, and in movement at the same time. As one is better calculated, the other becomes more and more fuzzy.
I think music, and listening to equipment is the same.
One can enjoy music, and gain an appreciation of the sound over a period of time. and come to make an opinion about it.
When the 'test' is paramount, and the switching includes having to make conscious decisions, that blocks a lot of the processing the brain normally does when listening for pleasure.

This is my belief.

My own method of listening to decide which sounds better. Is to just listen for an extended period of time(hour+) to one combo of stuff.. and note the freely rising perceptions in my mind concerning the presentation.
I can certainly remember those internal comments and feelings. Way better than any sort of 'aural memory'
And My method works for me when I audition gear.

[Calypte]

Frank, assuming I understand your procedure correctly when using the A-B switch, there's the fundamental problem in that you have no "control."  You are comparing one amp to the other with the switch in the circuit, so the switch is always in the circuit, and one of the amps under test is always in the circuit.  But you don't compare either switch position to no switch at all during the same listening session, and neither do you compare either amp to a third amp that's not being closely evaluated.  If you had a third amp, perhaps one that you already consider inferior, then when (if!) you hear a difference between either of the A-B amps and the third amp, then you have some assurance that, yes, today at least, your ears are still good.  Also, let's assume that you mentally tighten up a bit when using the switch, as compared to the more relaxed listening sessions where you listen for extended lengths of time.  You need some way, during those "relaxed" sessions, to have the amps switched in and out while you're not even aware that it's supposed to happen, while you still think you're doing one of those relaxed, extended sessions.  If a relaxed frame of mind is necessary for a good test, then you need to be able to take advantage of that situation without knowing that the amps are being swapped, or are being swapped with a third "control" amp.  Constructing a genuinely valid test that eliminates all forms of bias is difficult.  Also, as another poster has pointed out, it's possible that the mere act of performing a test alters the results.

[Wayner]

By your description of the experiment set-up, you left one item uncontrolled, and that would be the music itself. Did you restart the exact same piece of music when you changed playback devices?

If not, the experiment is not totally controlled as the music and it's sonic characteristics are in constant flux. However, using pink or white noise will not reveal any of the other sonic virtues, other then spectrum reproduction.

It would be like testing the clarity of 2 different pieces of glass, mounted side by side in a train car, and viewing the differences while the train is moving. In this case, the scenery constantly changes, preventing the viewer from making any real, scientific comparisons.

Wayner

[Wayner]

One other thought has crossed my mind and it deals with the source material you are using for the A/B test. Perhaps in the initial stages it should not be music. Music is too complex, and the brain fixes itself on several, observation obscuring functions. First, does the listener like the music? Are the rhythm patterns to the liking? We are then thinking about cord structure, harmony and all the other ingredients that go into music. There is just too many things to be exposed to when do this comparison.

A much better sources would be with musical instruments, but with extremely simplified content, like a single piano note stuck with the sustain pedal held. The listener then can listen to the dynamics of the hammer strike, the texture of the decay (along with the linearity) and so on. A single cymbal crash would also be very helpful, as would a single bass note strike.

These musical tones would then free the listener(s) from making judgements on musical style, content and complexity, and allow you to observe how a single instrument from initial play thru decay will behave, thru the selected device.

Cymbal crash and selected A component, then cymbal crash on selected B component, recording the results on paper will all help in doing a more controlled A/B test.

Wayner 

[rcag_ils]

If not, the experiment is not totally controlled as the music and it's sonic characteristics are in constant flux. However, using pink or white noise will not reveal any of the other sonic virtues, other then spectrum reproduction.

Are you saying when you feed the same musical passage through the same amp sounds different every time? I hope not. You are making this way to complicated.

One other thing Frank could do is purposely make the Fetvalve sound worse, by using the ever-popular audiophile tube rolling method, put a couple of weak or bad tubes in the Fetvalve, see if you can hear the diffenences. There shouldn't be any mental shut down then.