Subjective Loudspeaker Reviews - All About Price?

When taking on the task of reviewing products for consumers there are different ways to get across the conclusions and experiences of the reviewer. Objective testing and subjective viewpoints are both presented, and price is also included in the evaluation process. But there are so many methodologies and considerations, how does Audioholics keep it all straight? Frank asks a very valid question and we thought it was good enough to include it for public consumption to outline just how Audioholics handles this issue.

Frank: Dear Audioholics,

I have been reading your online reviews for a few years now and I am a fan of your approach to rating AV equipment and debunking outlandish vendor claims. I do think though that you could be a bit more forthcoming on rating your speakers (possibly other components as well). My beef is that you will give a performance rating on a product and then you qualify it by saying something to the effect of "if this is in your price range" or just "at this price point".

I agree that price is a very important consideration in buying AV equipment but should you not leave that up to the reader to decide? Would the reader not be better served by you giving an absolute performance rating against the same criteria for each type of product? I don't mind paying more money for a product if there is a corresponding quality improvement. It is difficult to discern this value when you do not know what the minimum performance to be expected is for any price point. Just a suggestion; keep up the good work!

- - - - - - - - - - - -

Dr. A: Hi Frank,

Thanks so much for your feedback, you make some good points. Let me try and at least clarify how and why we chose to do it the way we do. From the very beginning, Audioholics has consistently stated that all ratings are based around a product's price point. There is simply no way to have any sort of absolute scale when you are dealing with subjective reviews of, say, bookshelf speakers that range in price from $49 - $6000. The scale would be so terribly useless at that point as to be very unhelpful to the reader.

We figure that everyone has a budget. Though it may vary, everyone has an amount of money they are willing to spend on a piece of equipment. With that being the case, we have always maintained the (we believe) sensible process of factoring an AV product's price into the mix when calculating both the performance and value ratings to the consumer. For the most part we use the retail or MAP (minimum advertised) price, except in some situations where we know the MSRP is ridiculously or artificially overinflated compared to street prices.

If our reviews were completely objective (all measurements, no opinion) then I would wholeheartedly agree with your assessment - a completely across-the-board rating system would possibly make more sense. With the nature of subjective reviews, however, and having multiple reviewers, we feel that our caveats and value-based ratings are the best choice. Consumer Reports tries to do it the other way  (dividing up only by the type of speaker) and we feel they fail miserably at it due to their chosen testing methodologies and the fact that they completely avoid large cross sections of the industry (ever see a high-end/priced AV product reviewed on CR besides Bose?) To get an idea of how this might affect an organization's ability to provide meaningful reviews, currently CR has a grand total of just six brands of speaker involved in their loudspeaker testing (Sony, Polk, Bose, Infinity, JBL, and KLH).

I hope this helps at least clarify our position on this for you. Have a great week!