Subjective Loudspeaker Reviews - All About Price?
When taking on the task of reviewing products for consumers there are different ways to get across the conclusions and experiences of the reviewer. Objective testing and subjective viewpoints are both presented, and price is also included in the evaluation process. But there are so many methodologies and considerations, how does Audioholics keep it all straight? Frank asks a very valid question and we thought it was good enough to include it for public consumption to outline just how Audioholics handles this issue.
Frank: Dear Audioholics,
I have been reading your online reviews for a few years now and I am a fan of your approach to rating AV equipment and debunking outlandish vendor claims. I do think though that you could be a bit more forthcoming on rating your speakers (possibly other components as well). My beef is that you will give a performance rating on a product and then you qualify it by saying something to the effect of "if this is in your price range" or just "at this price point".
I agree that price is a very important consideration in buying AV equipment but should you not leave that up to the reader to decide? Would the reader not be better served by you giving an absolute performance rating against the same criteria for each type of product? I don't mind paying more money for a product if there is a corresponding quality improvement. It is difficult to discern this value when you do not know what the minimum performance to be expected is for any price point. Just a suggestion; keep up the good work!
- - - - - - - - - - - -
Dr. A: Hi Frank,
Thanks so much for your feedback, you make some good points. Let me try and at least clarify how and why we chose to do it the way we do. From the very beginning, Audioholics has consistently stated that all ratings are based around a product's price point. There is simply no way to have any sort of absolute scale when you are dealing with subjective reviews of, say, bookshelf speakers that range in price from $49 - $6000. The scale would be so terribly useless at that point as to be very unhelpful to the reader.
We figure that everyone has a budget. Though it may vary, everyone has an amount of money they are willing to spend on a piece of equipment. With that being the case, we have always maintained the (we believe) sensible process of factoring an AV product's price into the mix when calculating both the performance and value ratings to the consumer. For the most part we use the retail or MAP (minimum advertised) price, except in some situations where we know the MSRP is ridiculously or artificially overinflated compared to street prices.
If our reviews were completely objective (all measurements, no opinion) then I would wholeheartedly agree with your assessment - a completely across-the-board rating system would possibly make more sense. With the nature of subjective reviews, however, and having multiple reviewers, we feel that our caveats and value-based ratings are the best choice. Consumer Reports tries to do it the other way (dividing up only by the type of speaker) and we feel they fail miserably at it due to their chosen testing methodologies and the fact that they completely avoid large cross sections of the industry (ever see a high-end/priced AV product reviewed on CR besides Bose?) To get an idea of how this might affect an organization's ability to provide meaningful reviews, currently CR has a grand total of just six brands of speaker involved in their loudspeaker testing (Sony, Polk, Bose, Infinity, JBL, and KLH).
I hope this helps at least clarify our position on this for you. Have a great week!
Confused about what AV Gear to buy or how to set it up? Join our Exclusive Audioholics E-Book Membership Program!
Recent Forum Posts:
TLS Guy, post: 338569
It is true that the room does affect the perceived sound to a degree, and with far too many speakers to a high degree.
I would disagree with the term “to a degree.” The room is more important than the speakers in virtually every case I've ever encountered. I'm not suggesting room treatments are all that important. But the acoustic character of the room - it's dimensions basically - are more important than the speakers or any other part of an audio system. I can get better sound from lesser speakers in an acoustically superior room than I can with better speakers in an acoustically inferior room.
I did some blind testing years ago using two rooms. One was 25' long and 14' wide. By placing the speakers 6 feet from the back and listener 6 feet from the front and along the long walls, the acoustics were outstanding. The other room was 15' square. Neither room had any audiophile treatments.
Both had carpeted floors. The better speakers were B&W Matrix 802 and the lesser speakers were Boston Sub/Sat 6. All but one of the testers preferred the sub/sat 6 in the good room over the B&W in the poor room in a blind test. Yes, they preferred the B&W in the good room over the Boston in the good room. The difference in price between the speakers was pretty dramatic. So was the acoustic presentation produced by the rooms themselves.
One of the reasons movie theaters have good sound is that they are significantly deeper than they are wide.
The listening room itself is #1. Then speakers and then everything else.
This is all good stuff. I wish I knew what those wild screen speakers were - they looked like masts..completley thin and very cool.
Amazing what guys were working on back then (We all thought Bose 901's were good and like a Corvette, sorta expensive and beyond out reach)
JackVa1, post: 338764
Thank you ! I totally enjoyed reading the CR from 1976.
The AVID's were the speakers I bought and now passed to my son. Trust me, they were and are still good speakers no matter what you think of CR. I swear I blind tested them agains a pair of large ADVENTS, a weird speaker that looked like a large white screen, and a big speaker that sat in a corner with the midrange or tweaters actually aimed into the corner of the room (these suckers weighed a ton).
All were pricy.
I have little doubt that your Avid's sound good, and there are other speakers of that era that are still well regarded by many (the Yamaha NS1000 for example). But the fact is, knowledge about how to build a good speaker has advanced, so the number of very good speakers available thirty years ago has been greatly eclipsed by the number available today, and due to the information available to designers today, there are speakers available now that are superior to the best that was available then.
davidtwotrees, post: 338769
So, I disagree wholeheartedly that speakers cannot be scored.
The notion is not that speakers cannot be scored, but that to produce a very accurate and reliable score, more is needed than a simple in room RTA measurement and a panel of moderately, or non trained listeners.
TLS Guy, post: 338569
It is true that the room does affect the perceived sound to a degree, and with far too many speakers to a high degree. .
I bet that when the anechoic chamber FR is compared to the in room FR, there is more than just ‘a degree’ of difference in them. I wonder where that would come from?
davidtwotrees, post: 338769
I used to be a fairly serious wine drinker. The Wine Spectator had a one hundred point scale for any bottle of wine it reviewed. While it wasn't perfect, the numbers said some things pretty clearly to me a wine drinker.
A bottle of wine scoring in the eighties was a solid performer, one scoring in the nineties was something special, and a bottle that scored 98, 99, or the rare 100 was an exceptional bottle of wine that would knock your socks off.
If you start to include price in there, a bottle that scored 85 and cost $400 was a terrible value, while one that scored 85 and cost $12 was an exceptional value.
So, I disagree wholeheartedly that speakers cannot be scored. Yes, there is no perfect review methodology, and there are many variables. But as a consumer, I want the expert to give me his best judgement on a speaker's performance. Then tell me what it cost, and I as an informed consumer can make my personal value determination.
That makes two of us - give me a score and I'll use my ears and wallet (and wife) to select.
We need the governement to…..(J/K)