[1000a]

I guess many have seen this?   basicly comments about the integrity of reviews from the big mags with heavy advertizing involvement.   Its intersting to note for me cause when I came back to start getting some new gear in 2001 after sitting still with my gear forever, I ran into the TNT website (european with none, notta ad on its premisis just labors of love).  lots of reviews for me cause I took a sabitical.

recently making some purchases of gear not well covered on on AC or AA I was forced to try and trust commerical type reviews which I think are not that reliable, anyway I found some good I think places such as Bound for Sound (no ads) and such.  I love the TNT sight where I learned about Scott and Nels and there comments on audio gear (so I tend to still follow them-cause the sound they enjoy seems to be on my same page)

So: 1- who hear has spent some time on TNT?  Its a great sight.  2-what reviewers do you trust and like, what mags websites and such?  3- Stereofile dumping Audiogon, was sending people to say Promitheus and smaller heavy hitters not in their best interests?

FWIW i just try and study the reviewers gear, likes, dislikes, general tastes of presentation and such.  I recently discovered Lynn Olson who seems more than up and up.  Other than that I really hope to find low ad supported reports to no ad reports.

it seems no secret to me discounts are offerd to reviewers but who wants a discount on gear they don't like- no one.  but in the big mags there is presure I think to keep all the business relationships in good harmony so there is really never a rev. where we get some kinda reasonably objective comments, mush less subjective out of fear of the boss and industry banishment, plus no more sweat deals.  just wondering how you guys wade thru a system such as this?  plenty of good stuff gets no mention here or on AA.  not because its bad just because a ton of audio guys read and read and never post.

just tossing it out

http://www.high-endaudio.com/reviewers.html#Intro

[Kevin Haskins]

Back when I was an audiophile, and not a designer I used to pour over magazines and reviews for loudspeakers to audition.

Now that I understand much more about acoustics, acoustical engineering and have an insiders view of audio I tend to look for people who I know are using good methods.   I look for people who exhibit good design skills and who have good no B.S. knowledge and skills that they bring to the table.    I avoid many of the more common brands that are well reviewed by the audio press.   Many of them are better at marketing than they are at engineering.    Speaking of engineering... some people snub engineering as a valid method for designing loudspeakers.   I stay well away from those people.

I've also taken a BIG turn from being very much in the subjectivist camp and now I'm much more in the objectivist camp.   Don't get me wrong... the final judgement is in the listening but I've found that there are a number of variables that are very difficult to control, the room topping the list.  That and those who rely too much on listening don't understand all the fundemental underlying principles.  How do you improve something if you don't understand it?   Trial and error?   I believe in the scientific method and using logic and engineering to design loudspeakers.   I also believe that the final "touch" is somewhat of an art.   Its where you put your final signature on the design.

My advice is to listen to a lot of loudspeakers to find out what you like.   I'd join & go to DIY events, local audiophile clubs and in general hang out with audio dweebs.   You learn a lot from exposure about your likes and dislikes.   Take some reading time to learn about room acoustics.   Its time well spent and will reward you with much more value for your money than will playing with expensive tweaks.   

Most of all... trust your own judgement.   Don't let someone else tell you what good sound is.   Avoid letting reviewers, no matter how well respected, tell you what you should like.

[1000a]

audio dweebs. 

oh my God I never thought of myself with a term like that

I recently jumped deep into room treatments, it seems 50% of a stereo's potential is completely lost without the effort.  room 50%/stereo 50%

wow was I shocked - an untreated room with fantastic $$$ gear is like a horse at the derby w 2.5-3 legs max.  a treated room with good sounding well designed and choosen  gear I would take every time.  the room efforts on my part seemed to nearly double the level of quailty in MS.  DIYed thats one incredible return on the $$ spent.

the knowledge gained in studying the room and such has given me much better insight into how all the parts behave particularly the stuff on the rack, how can a cable be considered bright/dark non detailed or what not if its used in an untreated room, it seems no attention here is insane and skews all kinds of commentary on gear.

[Zero]

Ideally, a reviewer’s job is straight forward. In the perfect world, a reviewer should only evaluate electronics they can comfortably accommodate that is within the scope of their experience. In other words, you won’t hand a guy a $10,000 pair of speakers if his/her experience is primarily in the $3000 territory.

In the real world, listening bias and taste yields a major influence over whether or not we give a component the green light. Even the paid professionals sometimes fall into the trap of judging a component based not on the merits of its design, but strictly from a self-imposed reference of what good sound should be. At the end of the day, a review should be a synopsis of what a product is about, what it does well, and what it may not do so well.

Unfortunately thats not always how things go – especially when most of the reviewers out there are unpaid volunteers. When you go through in common variables of questionable experience, back round, listening bias, room issues, along with a host of unknowns – it becomes very difficult to gain valuable input from a good number of reviews out there.

In the end, people latch onto particular reviewers for a host of reasons. Onr may find a reviewer to be informative, entertaining, or even find the reviewers taste and listening habits fall in line with your own. Still, I concur with the basic primus that you should never let another person flat out tell you what to like and enjoy.

On a personal note; since I began dabbling in the world of reviewing earlier this year, I’ve become more and more attached to a section of my reviews I refer to as “the compromise”. This of course, is playing an incredibly dicey game with manufacturers who normally wouldn’t want to touch that type of thing with a ten foot poll.  However, the bottom line is that no product is perfect and there are almost always caveats one must consider before making a purchase. So long as people may be investing their hard earned cash on my word, I am certain to make sure there will be no surprises in store for them when they bring a product home.

Interestingly enough, taking this route also poses a problem to readers. Yes, readers! As it turns out, a very large number of people with audio nervosa _do not_ want to read comments that suggest a product has a weakness or two! A lot of people want to justify their purchase by bringing home “giant killers” that sport nothing but buzz and hype. But I digress;

Like everything in life, you have a choice. Do you wish to believe a reviewers word or don’t you? The criteria each reader uses to judge a reviewer varies significantly. The best advice I can offer is to determine what’s important to you in a review, and find someone that delivers on a consistent basis. 

[1000a]

thanks very helpful and enlightening, my grasp on the the first level of pitfalls is only the most obvious.

then there is all of this I have given little thought to, its really helpful and interesting.   I have read some reviews where I felt or suspected the reviewer had not much experience- but everyone has to start somewhere. 

So far I have been very happy with my selections, when it feels right to me I just go enjoy my music and stay off the forums and reviews completely.

[TONEPUB]

Well, the review thing is always very difficult. 

Granted there are all the conspiracy theories surrounding how things get done and there might be a little bit of monkey business here and there but in the end, if the reviews are false or misleading, they don't serve anyone: The manufacturer or the reader (or the credibility of a reviewer)

I can't speak for the other magazines first hand but everyone on the TONE staff owns all or most of their gear.  This is a business and a full time job, not something I'm doing to get free gear.  (none of the better mfrs. give their gear away anyway...)

We try very hard to describe the character of a piece of equipment accurately enough so that you can decide if it is worthy of your time to investigate.  If we write a fluffy review that doesn't make sense it fails everyone. 

You go to look at the gear, or worse just buy it based on a review, get it home and hate it and we all look bad. I want you to have equipment that YOU enjoy when you listen to your music.  I have a system that I love, but it probably isn't for everyone.  However, it is a fairly accurate tool that I can use to evaluate other components.  For what it's worth, I have had my fair share of people both in and out of the hifi industry stop by and give me the atta boy on my system, so I feel pretty comfortable about the choices I've made.

That is the same reason we don't do editors choice awards or have large lists of recommended components.  You can't always take a Class a amplifier, class b cd player and a pair of class B speakers and be guaranteed of good results.

The point of reviewing gear that you own is well taken.  I've done a little of that myself, but some of it was during the process of upgrading to the system I own now, so I guess guilty as charged there.

We have also made it a policy to make the mix of gear reviewed by advertisers and non advertisers be as close to 50-50 as we can.  Same with our year end exceptional value awards.  We've never turned anyone away because they don't advertise with us. And we've probably picked up half of our advertisers because they were people that said "I'll send you a piece of mine to review, but I WON'T advertise with you."  Then four months after the review came out and they made some sales because of our review, they came back.

I believe that all of the people I've met here at AC and elsewhere are intelligent enough that they are not going to buy a pair of speakers just because I say so...

It really is about the journey for me.  It's a great day for me when someone calls or sends an email and tells me that they read one of our reviews, went to listen to the product, liked it enough to buy it and felt that our assessment of that product was right on the nose.  Everybody wins then.  I look like I know what I'm talking about, the reader gets a great piece of gear and the mfr sells a piece of gear to a happy customer.

Honestly, folks that's what gets me out of bed in the morning and why I love my job so much.  And I get a lot of these phone calls, so I feel that we are on the right track.

Again, I'm not telling anyone to just buy something because I like it.  If we were having a drink at a bar, and you asked me what speakers to buy, I would ask you a whole bunch of questions about your system, your music, your room, how much time you have to listen and how much money you have to spend.  Then I would suggest about five things for you to audition.... 

I've got the gear I want and I'm thrilled.  When I listen to my system, I'm totally digging the music.  I've had that experience every where from 2500 bucks to well over 100k.  I'm just trying to help you have a good time too!

But again, all I can do is make suggestions.  You still have to check it out for yourself.  I just hope that we've been able to help.  I also hope that it gives you a little more insight into what we are trying to accomplish.

We appreciate you reading, and we'll keep trying our best!

[Ethan Winer]

Good thread folks.

I especially enjoyed this comment:

I recently jumped deep into room treatments, it seems 50% of a stereo's potential is completely lost without the effort.  room 50%/stereo 50% wow was I shocked - an untreated room with fantastic $$$ gear is like a horse at the derby w 2.5-3 legs max.  a treated room with good sounding well designed and choosen  gear I would take every time.

This is why I find it annoying when some people tout unmeasurable, unproven "tweaks" claiming a huge improvement, all the while ignoring room treatment which actually does make a huge improvement.

--Ethan

[mfsoa]

And how about the reviews of room treatments where it turns out the reviewer simply has none to begin with?   

Sad and funny at the same time.

[TONEPUB]

Ethan:

I couldn't agree with you more!  I've spent about a thousand bucks on some bass traps and a few
panels from GiK, along with a little bit of sonex and a few rugs for my floor and it has made a huge
difference!

I always tell people that while some of the premium cables and such will make an incremental
improvement, that's the LAST thing to go for.

I think a little bit of room treatment offers the biggest bang for the buck out there!!

Perhaps buying a pair of bass traps isn't as much fun to talk about over dinner as the
latest cool 2000 dollar interconnect?

Interestingly enough, I found that the increased resolution of treating the room has helped
me to hear the nuances between various cables, etc much easier!!

[1000a]

Good thread folks.

I especially enjoyed this comment:

I recently jumped deep into room treatments, it seems 50% of a stereo's potential is completely lost without the effort.  room 50%/stereo 50% wow was I shocked - an untreated room with fantastic $$$ gear is like a horse at the derby w 2.5-3 legs max.  a treated room with good sounding well designed and choosen  gear I would take every time.

This is why I find it annoying when some people tout unmeasurable, unproven "tweaks" claiming a huge improvement, all the while ignoring room treatment which actually does make a huge improvement.

--Ethan

Thanks Ethan

its only with the help and general encouragement from yourself and people like you that I started into it.  My room stuff still needs attention in the looks department for sure, I think this is probably why people do not go down this road the thought of all the extra things and no knowledge of the improvements.  Once I heard the improvements-massive.

then I came to- well I can make it pretty or just make it moble the gains are so ridiculus carrying and placing in the room for a sit down is a no brainer for me.

[1000a]

And how about the reviews of room treatments where it turns out the reviewer simply has none to begin with?   

Sad and funny at the same time.

that is the strangest thing of all a pro who uses no room treatments, and I want guidance from these people

thats really pita-full, think about it, pros paid or unpaid with 1/2 strung tennis rackets?
What............

check the Greek audiophile video, 150-250,000. stereos no notta bass trap in site, unless I overlooked em.

[TONEPUB]

We have a couple of good freinds that are recording and mastering
engineers, as well as an audiologist that has been helping us with
our rooms.  I've been excited that they all have been pretty much in
agreement on how to proceed and we've learned a lot!

Even if you don't have access to the pros, there is a lot of good
documentation out there to help you dial in your room...

I'm very fortunate that I have a dedicated room, otherwise some of
the room treatments would have been a hard sell!

[Housteau]

I generally look for two things. 

First off, I like to know the individuals frame of reference.  This hold true for participants on audio forums as well and not just for ‘professional’ reviewers.  What does their listening room look like?  Is it a dedicated space?  Does it look to have good acoustics with the proper room treatments?  How is the speaker / listener interface set-up?  Is there a giant TV between the speakers?  Is it a home theater, or dedicated two channel set-up? 

How often have you read posts from individuals that sound very convincing and are well put together, then you have the chance to see their listening room and habits, then really wonder and question if they really have any clue at all?  If the good acoustics are not there, owning great gear will not matter nearly as much.

The second thing I look for is a track record of former reviews, especially if I have owned, or am very familiar with that item.  If so, did I find the same things true as they did?  If not, can I at least understand where he was coming from and agree to disagree due to individual preferences? 

[Nels Ferre]

Ideally, a reviewer’s job is straight forward. In the perfect world, a reviewer should only evaluate electronics they can comfortably accommodate that is within the scope of their experience. In other words, you won’t hand a guy a $10,000 pair of speakers if his/her experience is primarily in the $3000 territory.

Here is another viewpoint. When I started reviewing, it was very difficult to secure review pieces, even the low cost ones. Nels who? You write for who?  We will call you back. Some did- most didn't. Some agreed to a review, and the sample never arrived. Now, I can pretty much get whatever, within reason. Currently in house is approximately $14,000 worth of gear, that is getting boxed up and shipped back- not because I don't like it, but because you can't buy everything.

With your $10,000 speaker example, it doesn't mean the reviewer has limited experience with speakers at that price range, it could mean that no manufacturer offered (or sent)  speakers in that range before. Just because the reviewer hasn't written about $10,000 speakers doesn't mean he has limited experience with them.  That is an assumption.

So long as people may be investing their hard earned cash on my word, I am certain to make sure there will be no surprises in store for them when they bring a product home.

Well said.

[TONEPUB]

That's another excellent point, familiarity with what you listen to...

If you are constantly flipping gear in a reference system, it's hard
(if not impossible) for it to be a reference.

[1000a]

Thanks for the posts very helpful.  I hope this thread stimulates a lot of thinking and interesting conversation benifiting us all.

[Philistine]

Recently I've bought a replacement digital camera and wireless router, I used websites such as Amazon/Cnet/dpreview to make a mental shortlist but found customer reviews to be more useful.  Owners tend to be more open about the positives and negatives about what they own, this is not just related to low ticket items - car owners also freely comment about whats good and bad about their purchases.
I treat audio reviews as pure entertainment, I rarely see a bad review - at the same time we audiophiles are reluctant to chime in and say a piece of kit sucks, what I see is "this is a wonderful amp/CDP/speakers etc" and a few weeks later it's up for sale!  The downside of posting a negative comment is that flame wars are started and the second hand value could be discounted, so why bother as an owner to go down this path? 
So putting these together I go back to my view that audio reviews are pure entertainment and not to be taken too seriously.
 

[TONEPUB]

As someone who used to review cameras for Cnet, MacWorld and quite a
few of the other majors, they never liked us to point out the negatives.

Now that we are getting TONE photo back on its feet, I am getting very
excited about that.  As a professional photographer, I look for different
things in a camera than the amateur user for different reasons.

However, I'm most concerned in what I call the 5-minute test, because
most people dont want to read the manual. Hence a piece of gear has
to be intuitive and easy to use first and foremost.

I've always been more interested in ease of use and image performance
first and foremost, with feature set later, but an amateur user is usually
more concerned with the doodads.

Also, different photographers have different needs.  Someone who shoots
weddings needs a different feature matrix than someone who shoots products
in the studio or a landscape photographer.

The problem with many of the magazines is that they don't have any
pro shooters on the staff with enough seat time with this stuff.  Believe
it or not, when you get a new camera from Nikon, Canon, etc. they all
come with a big packet of "reviewer guidelines" that are essentially a review
in a can written by the PR dept that has no idea what to do with this stuff
in the first place....

[Kevin Haskins]

Well, the review thing is always very difficult. 

Granted there are all the conspiracy theories surrounding how things get done and there might be a little bit of monkey business here and there but in the end, if the reviews are false or misleading, they don't serve anyone: The manufacturer or the reader (or the credibility of a reviewer)

You have a tough job.   Audio is like politics and religion, your never going to get a consensus.   

I'm pretty hard on reviewers in general but I wouldn't want the job.   In terms of publication, I enjoy reading Tone.   Your about the only publication I read on regular basis that is  non-engineering related.

[TONEPUB]

Thanks for the kind words.

I always tell people it's like buying Christmas presents for people. If you are
a good present buyer, you get your fam & friends things THEY want, not
what you want!

For me, it's all about you getting the sound you want.  I'm just trying to help
you find it!!

Thanks for reading, we all appreciate it!