Tire Testing Without a Pressure Gauge!
From Audio Basics, Volume Thirteen, Number One, January, 1994
Occasionally Car and Driver or Road and Track (and even Consumer Reports for that matter) runs a long article comparing the road handling, ride, and durability of different brands of tires. What if they used experienced professional English Majors for testers, with all the tires mounted on really expensive cars (but with a random assortment of really expensive cars of different makes and models)? What if none involved with the test procedure knew about tire pressure gauges or wheel balancers so that the tires were simply mounted with random amounts of air pressure and no wheel balancing?
Could the testers each write earnest and eloquent long, detailed, and interesting essays about their feelings regarding each set of tires they tried? Sure they could.
Could a tester come to the conclusion that he found one set of tires that he really liked best of all? Certainly he could. Could a different tester reach a different conclusion? Absolutely, especially if his own ideals regarding car ride and handling had different priorities (one may prefer firm and precise, another soft and isolated).
Would any of the conclusions reached be valid regarding the actual desirability of the brands of tires tested? Of course not - because there were significant uncontrolled variables such as tire pressure, tire balance, basic suitability of the tire for the vehicle, and the testers' education and training regarding the objective engineering aspects of tire design, construction, and testing instrumentation - you need to know how to run the balancing machine and read the pressure gauge too.
In fact, the person liking a soft and isolated feel might actually pick a harsh racing tire as his favorite if it had randomly been inflated to only about 10 pounds of air and was by chance in perfect balance without needing wheel weights and installed on a big old Buick. Meanwhile, if that old rayon cross-ply $10 farm wagon special had happened to be inflated to 50 pounds and put by chance on a Corvette, the driver might think it had the most crisp performance of all.
Even if the drivers were all professional automobile testers and only one brand of car was used for the testing (one noted for excellent suspension design), it would be easy to fool the testers into picking the "wrong" tire if the inflation pressures were messed with to set one brand just right, but all the others a little off. A great racing driver probably would likely know that the pressures were wrong right away, but a great racing driver is off racing, not writing up tire tests.
The point is, of course, that if you knew that the tire tests had been done without a pressure gauge, you would never pay serious attention to the results. An "I like it," in the absence of reasonable objective documentation to show that major variables were accounted for is not a valid test at all. For that reason, the car magazines do use tire pressure gauges and wheel balancers and do explain their test procedures in detail to you. They want you to understand that they are trying to provide you useful and repeatable information along with the entertainment.
Why then do we pay such great homage to the eloquent purple prose of many esoteric audiophile magazine reports? When Shun Mook discs get pages of happy prose, when $5000 triode vacuum tube amplifiers are described in glowing detail (a pun - ha ha), when wordsmiths worship wild wandering woven wires, but there is no serious attempt at objective instrumentation, something has gone wrong.
Why don't we just observe that they are not using a tire pressure gauge, and give the reports the respect they deserve? Remember, the guy on the back of the bus with the boom-box turned up to full distort can also report, "I like it," and his "I like it" in the absence of objective data is just as valid as your opinion (or the tire tester's).
Note too, that the modern world likes to deal in opinions, not information. The reporter asks the man on the street, "Do you think we should intervene in Bosnia?" not, "Do you understand what the underlying causes of the conflict in the Balkans are all about?" it is much more fun to deal in opinions than in knowledge. The observation that opinions, in the absence of knowledge, are not very useful is immaterial. Opinions are much more comforting to deal with because nobody can prove you are wrong. You are "entitled" to your opinion, so there, knowledge and facts be dammed.
Note that I have no axe to grind regarding any specific review of any of our products. Over the years our products have been reviewed in many different magazines, almost always fairly and usually positively. Interestingly enough, the magazines that give the least encouragement to frivolous expensive accessories that can only be justified subjectively tend to pay the most attention to our designs. Our products are designed to not screw up the source material and to be as ideal an interface with whatever else is in the system as possible. Thus they tend to work well in a wide variety of applications, and thus tend to be liked, no matter where or how they are used. The reviews usually confirm this.
We wish however, that the review process in general could become more objective. The following are our suggestions as to how this might be accomplished using some of the methodologies that have evolved here to help keep us honest in the design evaluation process.
First, make sure your tests are useful.
The major reason that too much of the subjective process of high fidelity evaluation and reporting has drifted into a La-La Land of authoritative, "I like it and therefore it is just wonderful" evaluation, complete with its own language and culture, is that the mainstream objective measurement process has failed.
The process of providing commercial specifications has deteriorated into a mutual admiration society of off-shore appliance manufacturers, all endeavoring to show that all the plastic molded noise reproducing appliances they stamp out measure just wonderful, and thus must sound just wonderful. Certainly the purveyors of myriads of push-buttons, blinking lights, and "more is better" for the masses would never want to convey the concept that you might want to consider whether or not their products actually had a chance of reproducing music. That was not the point of their design or production. They were designed to enhance your life-style, just as the shop on TV fake diamonds and stair steps do.
The specifications and test procedures have been established and executed by those that seem to have little interest in the actual high fidelity reproduction of music at all. Perhaps unconsciously, they even seem to set up standards and conditions of test to ensure that any real differences they overlook are swamped out. For example, when I visited Julian Hirsch's facility several years ago, I noticed that his test lab was a converted attached double garage. The floor was hard, the environment was metal test benches and racks. The room had the sonic character of a grade school cafeteria. In his lab I could hear no difference at all between any amplifier or preamp he used, including ours. He had both big B&W and KEF speakers set up side by side. I had no clue which I liked better. All components that measure good sound the same - true in Hirsch's test environment - a self fulfilling closed loop.
Introducing new objective data into that environment was not helpful. My engineer, Aado Perandi, demonstrated to Mr. Hirsch that two amplifiers that measured the same under IHF standards could show quite different characteristics when scrutinized a bit more closely. We showed that if one took a typical amplifier and drove it into a reactive (capacitive) load with a fast transient, that the usual result was a leading edge overshoot on the output wave form actually a damped internal oscillation - lots of high frequency distortion. We then moved the scope probes from the output of the amplifier under test to the input! The leading edge ringing overshoot showed up there too! The distortion at the output was reflected back to the input - and even showed up at the input of the other channel, which was not being driven into the tough load. This meant of course that the distortion was being reflected through the power supply and that the source driving the amplifier was now being screwed up too. There was more going wrong than what the standard and simple IHF "black box" tests were showing. We also demonstrated that with our amplifier of that vintage, the output overshoot was much smaller (the circuit much better coped with the reactive load) and more importantly, there was no reflection at all back to the input or to the other channel. Our amplifier was internally stable into the reactive load and was not loading down the driving source.
Note that since the distortion generated by an amplifier will likely show up at its input in addition to at its output, that the standard A-B test procedure of driving two amplifiers at the same time from a common source and then quickly switching the speaker load from one to the other is invalid. Since the test set-up essential ties the two amplifiers together at their inputs through the common source, the distortion at the input of each amplifier will actually be the sum of the distortions of both. Both amplifiers will likely "sound the same" under this set of conditions (and likely you will go away wondering why nothing really sounded very good). We are amazed that this invalid procedure continues to be used and that its limitations continue to buffalo even the golden ear reviewers.
Anyway, Aado and I were able to conclusively demonstrate to Mr. Hirsch that a simple test could show that amplifiers that "measure the same" under easy IHF conditions do not necessarily behave the same under other conditions.
The difference in distortion was very significant. We had hoped our demonstration would motivate the tester into looking further into improving the test evaluation process. Unfortunately, we were told that since our innovative methodology was not part of the published standards, that it would be unfair to subject products to a more demanding scrutiny. He did not make up the rules, he could only go by them.
This isn't a game. The search for knowledge isn't limited by "the rules." If the object is to understand what high fidelity is all about then one must welcome and evaluate new ideas and evaluation techniques.
One must understand that a "set in concrete" old set of test standards may not provide all the data necessary. For example, a tire pressure gauge, balancing machine, alignment rack, and even a tire rounding lathe and careful use of all this equipment would not identify a useless radial tire with an out-of-true internal circumferential tread belt. Even though the tire would test perfectly with all the equipment above, it would still vibrate like crazy (behave like it is badly out of balance) when actually driven on the road because the tread would try and follow the wobbly internal tread belt and shake back and forth. Only if one set up new and better objective test standards (and used an X-ray machine to look for the crooked tread belt) would one find out that the tire really did not test perfectly.
Two amplifiers with widely different amounts and spectrums of distortion do not sound the same. For example, if one amplifier is load sensitive, and shuts off or melts when connected to a reactive speaker cable, and a different design is able to cope easily with the strange cable load, you will hear the difference. One amplifier is shut down, the other is still playing the music. The IHF tests were inadequate - they rated both amps the same and they obviously were not.
When the old line engineers and technicians try to tell us that the tire tests perfectly, and thus it must work perfectly, and we can feel it shaking away when we drive it anyway, it is time to improve the test methodology.
You do not fix the problem by throwing away the pressure gauge!
Every time I see the movie, Star Wars, and it comes to the scene in which Luke Skywalker is making his bombing run on the Death Star, I wince. For there is good old Luke pushing away the sophisticated electronic targeting computer and getting ready to aim his missiles at 2000 mph into a I foot hole by eyeball. Sure! Did you know that during the course of World War 11, not a single high altitude horizontal bomber ever hit a single moving ship at sea with a single bomb! The dive bombers and torpedo planes, operating at very close range, did all the damage. Sorry, but human beings simply cannot "eyeball" precisely and repeatably and the bomb sights of the era were too crude to be of much help. Actually, it was not until the Gulf War that our weapons got smart enough, but by Luke's time the targeting computer should have been a bit better yet. Anyway, the reason I wince is that the movie reinforces the old adage that technology is bad and that we can eyeball it through with better and more consistent results.
You hear differences between amplifiers but the commercial tests report that the differences do not exist, well then let's not do the tests, let's just listen. We know what we like and only our feelings about the equipment are important. The test process (technology) is bad, so we will do it Luke's way. Throw away the pressure gauge, we can run the tire tests better by ear. Not hardly!
The "science is bad, keep the faith and just listen" approach to understanding high fidelity leaves one open to every outrageous sucker claim that comes along. if one does not understand that it is possible to calculate and measure the surface area and the resonant frequency of a given bit of damping material and thus determine its acoustic efficiency, one can embrace a little $500 dot that claims to make major improvements in your acoustic environment. Night events at spectator stadiums are full of the winking lights from folks trying to get the big picture from the stands with their little 10 foot range flash cameras. The fact that the light output from the flash falls off with the cube of the distances means nothing to them (nor do the blank pictures they get). To have daylight in the stadium, you need the sun (or high speed film, all the stadium's floodlights, a fast lens, a tripod, and a cable release). That little $500 sound dot will work just as efficiently as your flashbulb at the stadium, but unfortunately, a lot more expensively. I know, but it sounds just wonderful - no thanks, Luke. Spend the $500 on season tickets to your local concert hall - it will sound even more wonderful yet (as long as they keep the PA system turned off).
If you don't do the math to divide the clock frequency of a CD player by the frequency of interest, you will not gain the knowledge that a 1-bit CD player cannot retrieve all of the information from a CD until the crystal clock gets up to about 1,320,000,000 Hz (1.3 GHz, a speed Cray would kill for). With the fastest 1 bit converters out there now, running at 90,000,000 Hz, all one can retrieve is 4,500 of the 66,000 samples per cycle available on the CD at high frequencies. That is not high fidelity. In baseball it would be a batting average of .068, not even enough to make the little league. Don't talk to me about how wonderful your 1 bit DAC sounds, when its sounds have nothing to do with the information content of the source disc.
The "eyeball" method alone won't give you consistent results. It is the method of choice however by those who suggest that it must be very, very expensive to sound really good (especially those who are promoting and selling that very expensive equipment). Subjective claims, not supported by any rational objective data, tend to be a bit pricy. Remember, 1% tolerance precision low noise metal film RN60C military spec. resistors cost all of $.07 each in small production lots. If you had an experienced materials engineer estimate the cost to build that pet $5000 vacuum tube amplifier you would be in for a shock (and not from its power supply).
If the current set of test standards do not consistently predict the suitability of an audio component for accurately reproducing music, it is time to improve the test procedures, not to ditch the targeting computer completely.
So What Do We Know and What Needs Fixing?
In spite of narrow technophile claims that "all amplifiers sound the same" if they test the same under IHF test conditions, we certainly can hear differences in various audio playback systems. Either we are hearing things that do not exist (and thus we need fixing) or the test procedures are missing problems that do exist (and thus the test procedures need fixing).
Actually the attitudes of both the technophile who is a slave to obsolete measurement standards and does not listen, and the audiophlake who never measures but absolutely knows what he likes when he listens need adjusting. Neither approach is bringing us better audio equipment.
The wonders of technology are bringing us 8 bit resolution bit-stream CD players to replace the 16 bit units that previously could retrieve the data better. It also supplies 4-bit resolution DCC and Mini-Discs to enhance our life-style even more. It gives us overload prone wireless microphones to make live performances sound just like our Mini-Disc player, and provides multitudes of computer controlled electronic synthesizers to eliminate the musician completely. We are encouraged to buy little swivel mounted plastic speakers that will kill a chicken at 50 paces and to go into the speaker design business for ourselves by getting a top and midrange speaker from here and sub-woofer from there and getting them matched up all by ourselves. All along the way to better living through more electronic gadgets the technophile reassures us that it all tests just fine and thus sounds just fine too.
Meanwhile, the association of audiotweaks has brought us the joys of $500 per foot speaker leads, interconnect cables full of water, $300 each thumbnail sized dots to enhance your listening room, panty-hose for your speaker wires, $5000 20 watt tube amps, $10,000 external DACS (to replace a part the size of your fingertip that costs $20), digital clocks to make your electricity all better, gadgets to "break-in" your cables (gotta plug them in for a few days before you dare use them or the sound will be all ruined), and freezer pouches, marking pens, and a variety of clamps, weights, and elixirs to subjugate your CDs to.
In addition we are exposed to the sage advice that we really need two sets of $500 per foot speaker wires per channel, that we really need to eliminate almost all that nasty and bad sounding $500 per foot speaker wire and use a half a mile of much more wonderful sounding $1000 per foot interconnect cable instead, that we really need two amplifiers per speaker, that we really need eight channels - not two, that we need little IC bass boost boxes to make our $5000/pair speakers work right, that we need to destroy our floors and shelves with railroad spikes sticking out the bottom of most all our components, and that we need to subsidize the electric company and aid the coal miners and OPEC by leaving everything turned on all the time (this may work for our blenders and microwaves too). We are encouraged to risk electrocution with huge metal external speaker terminals and by plugging our exotic amplifiers into US 240V lines (putting 120V AC live on the chassis).
You can easily spend your entire hi-fi budget on all the life-style enhancing and magical mystical accessories and not have any money left over to buy the components at all. $2000 should buy you a great, long term keeper of a high fidelity system. You can spend it all on just the cables and not even be able to afford the cable break-in machine - wouldn't that be awful?
Meanwhile, the next month's magazines arrive and inform you on one hand that your system is obsolete because it does not provide for 16 x 9 ratio TV, ten speakers, and wireless infrared remote control of bass and treble settings of each, or on the other hand that your $5000 external DAC has been superseded by the obviously better $10,000 model, but only if you use the two foot in diameter propane filled interconnect cable (made of pure glow~in-the~dark irradiated copper fresh from the control room at Chernobyl).
There is obviously a lot that needs fixing here. It actually is possible to think rationally and separate out the ridiculous from the simply improbable but possible. But you must want to know what is happening, not just "do I like it?" We will talk more about it next month.
Frank Van Alstine
Copyright, Audio by Van Alstine, Inc., 1994. No part of AUDIO BASICS may be reprinted or reproduced in any manner without the permission of the publisher.
