[neekomax]

No idea which circle this goes in, but oh well...

So I'm watching this Dr. Floyd Toole lecture from 2015, and this little part https://www.youtube.com/watch?v=zrpUDuUtxPM&t=177s gets me wondering whether it is (and/or when it will be) possible to realistically attempt an audio version of the Turing test (that concept of which I'm sure most here are aware). In audio terms, obviously, one would have to subtitute audio fidelity for the classic Turing criterion of believable conversation with an artificial intelligence.

So say, a human voice or violin, or what have you. System and 'real source' behind screens, instant listener voting decides it.

Has this been attempted? Could it be? If not, why? Seems like an interesting potential benchmark with regard to the question of high fidelity that might cut to the heart of the matter and 'split the baby' (as it were) between subjective listening tests vs. ABX or meaurement based approaches. See which system can REALLY PERSUADE in the moment.

After all, isn't that goal of all this mishegas?

Make it like F1 (minus prize the prize cash in the gazillions  : Invite Wilson, VSA, Spatial, KEF, Salk, Andrew Jones, Toole, Duke, whichever crazy Finn at Genelec, AJin FLa, tutti quanti, to design their best shit, and see if they can fool 1... 5... 10%... 51%?... of listeners into thinking they're hearing the real thing.

Or maybe it's old hat now, and I'm just uninformed. But I wonder....

[JLM]

Marketeers are scared of scientific comparative testing, thus almost none have been published.  This gets into the debate regarding the validity of double blind testing, which is brought up routinely:  what ancillary gear to be used?  what is the ideal room to be used?  who will the panel of golden ears consist of and how will they be selected?  what music/recordings will be used?  what criteria will be used?  If comparing to live (unamplified) performances, the complexity goes up further and from my readings no audio system has succeeded in matching.

Harmon International (where Toole retired from) has a pretty cool setup for blind speaker comparisons:  decent room, visual screen separating listeners from up to 4 pair of speakers that via a remote controlled trolley system can be moved in and out of place.  An operator randomly switches speakers for evaluation and comparison purposes.  As I've read exhaustive testing does take quite a while to complete.  Recommend Toole's "Sound Reproduction" to learn the basics of room acoustics.

What is the goal indeed?  Most would say perfect reproduction of the original performance.  But in whose ears?  Start with the recording staff.  Next is the mixing staff.  Then the mastering staff.  And along the way is the producer and artist trying to influence the final product.  Then on to the audiophile, who have various rooms, sonic preferences, and gear.  Honestly mostly audiophiles don't want accuracy, they simply want to be entertained - whatever that means, beyond the gear hunt/lust. 

[Speedskater]

Marketeers are scared of scientific comparative testing, thus almost none have been published.  This gets into the debate regarding the validity of double blind testing, which is brought up routinely:  what ancillary gear to be used?  what is the ideal room to be used?  who will the panel of golden ears consist of and how will they be selected?  what music/recordings will be used?  what criteria will be used? ................... 

All those questions are raised by those that have a vested interest in double blind tests not working.
But the answer is simple. Start with one listener who makes all the choices. If a difference is heard, why then you can slowly expand the testing.

[neekomax]

What if it's not golden ears deciding what 'sounds better', but an entire audience simply voting on, say, which they thought was the person speaking an identical phrase as the sample being played from a speaker? Nothing fancy, and indeed, a speaker company could participate and convince SOME of the audience and not look terrible from a reputation perspective.

If the test is conducted on multiple systems in the same venue with the same audience, then it isn't about the room being ideal for music, it's about the speakers being convincing at a certain task that relates to music reproduction directly. In other words, quantifying something meaningful in a reasonable way that doesn't rely on special Hifi priests to deem things good or bad.   

[BobRex]

Isn't this is pretty much what Brian Cheney (VMPS) was trying to do?

[dB Cooper]

AR used to conduct 'live vs recorded' demos in the 1950's-60's that were claimed to prove to attendees that the difference between live and recorded was inaudible. They used a string quartet for the demos IIRC. This would have been analog tape, probably tube electronics, and pre-NR. So I guess they could have just stopped there technologically. Of course, the musicians had to do things like soften their attacks on notes so as not to overload the signal chain anywhere, but what the hey, it's all in a day's bamboozling. And of course who can forget "Is it live or is it Memorex".

None of these tests, of course, were blind- or double-blind. Many audiophiles shun such tests because so many 'night and day' differences don't make it through. Most of the reviews you read involve no objective testing, just someone who has figured out how to monetize their ability to write about audio gear. Neither tells the whole story, and the best reviews usually involve both to some degree (innerfidelity comes to mind).

[Letitroll98]

The human voice doesn't have enough range to be a good test of audio quality.  Need a broader octave range and wider dynamic range.  A piano comes to mind.

[neekomax]

The human voice doesn't have enough range to be a good test of audio quality.  Need a broader octave range and wider dynamic range.  A piano comes to mind.

Exactly, if it was something with a limited dynamic and frequency range like a human voice, the test should be easier for the manufacturers to do better better at, and therefore less scary for them. Surely, a good system can sound realistic enough to mimic short phrases of human speech, well enough to fool a certain part of an audience.

The top scorers could move on to a finals that could involve an instrument or two.

[kingdeezie]

Marketeers are scared of scientific comparative testing, thus almost none have been published. 

I would disagree that this is a factor. Consumers who want objective data scientifically proving that certain audio equipment and tweaks do or do not sound better or make a difference, generally already adhere to those conclusions which are already thought to be true. Its the reason why most cable conversations quickly end up in the intergalactic waste bin.

The issue with audio is that sound is highly objective. Can you scientifically prove what is the best tasting beer? The finest wine? The greatest cheeseburger? Soda? Art? Best car? Best picture quality? Impossible. Whenever you add individual human perception into the mix, you create an unscientific variable.

Despite what the science might say about audio, there is always going to be people who prefer something else, or hear a difference where there may scientifically not be one.

Perception is reality, and if a customer hears a difference, and enjoys it, despite what the numbers say, they are going to buy it. 

[charmerci]

I would disagree that this is a factor. Consumers who want objective data scientifically proving that certain audio equipment and tweaks do or do not sound better or make a difference, generally already adhere to those conclusions which are already thought to be true. Its the reason why most cable conversations quickly end up in the intergalactic waste bin.

The issue with audio is that sound is highly objective. Can you scientifically prove what is the best tasting beer? The finest wine? The greatest cheeseburger? Soda? Art? Best car? Best picture quality? Impossible. Whenever you add individual human perception into the mix, you create an unscientific variable.

Despite what the science might say about audio, there is always going to be people who prefer something else, or hear a difference where there may scientifically not be one.

I disagree - go over to the AVS forum. You'll get a bunch of people saying that all amps/pre-amps/DAC's sound the same (under double-blind.)

Objective? Easy, how many times have you heard music in the next room and knowing instantly it's a stereo or live music. Easy, peazy.

The preferences that people have are for differing sounding reproductions that don't meet live's criteria.

[Elizabeth]

Seems two sorts of folks.
Described elsewhere are thinkers and romanticists
Thinkers want fact and proof
Romantics want a story and what seems good.
Most of the posters here are wanting proof and facts, or does not count.
The romantics just want it to sound good.

The thinkers think the romantics are all fools.

[Letitroll98]

Such a nice way to say it Elizabeth.
And we actually do have cable debates that remain civil.  I can attest to that personally.  It's due to the quality of our membership, they'll argue points without making it personal.