Alan Sircom

I don't recall anyone ever calling a $5,000 audio product 'budget'. A 'bargain' perhaps, but not 'budget'. Criticizing the reviewers for things they have done wrong is entirely fair; calling them out over things you imagine they have done is not. The reason why the 'bargain' is often discussed in isolation largely comes down to the mercurial nature of the buyer. Product A is a good $500 DAC. Product B is a 'bargain' $250 DAC that performs almost as well as Product A. Mentioning that in print means those already owning Product A immediately put their DAC up for sale only to find their $500 DAC has a resale value of about $100, rather than about $250-$300 as a result of the review and the market being temporarily flooded. The reaction to such a statement is generally short-lived and the manufacturer of Product A tends to ride out the storm (these things tend to rebalance over the course of a five-year product life cycle), but owners go thermonuclear, and never forgive you (I still get called a 'turncoat' by some UK turntable owners over a review I wrote... in 1992). Once again, you seem fixated on the idea of the reviewer using his or her personal buying power to define a product's worth. Ask any reviewer or editor about this, and you have just perfectly described the #1 Bad Reviewer mistake - not everyone has the same budget as you. A product's worth is defined by how it performs in the open market. It is set in the context of who is likely to buy such a device category, but its main parameters are laid out by its performance compared to rivals. So, in the George Soros case you make, yes... he may very well find a $50k Volvo a bargain. And no, not everyone has George Soros' bank balance. But if you are writing a review of a product aimed at George Soros, you need to learn to think like you did have George Soros' bank balance (#1 Bad Reviewer mistake, remember?) and under those conditions, if you have a $50k Volvo that you would consider a 'bargain' in context, then surely it is what it is. We cannot arbitrarily set price limits for reviews, because the range of people reading the sites and mags is so wide. We're reasonably confident that the lowest end of the audio market today largely bases buying decisions on factors like price and looks rather than specifications, measurement or listening tests, but it's conceivable that people who visit this site might have anything from a couple hundred dollars worth of equipment right up to tens or even hundreds of thousands invested in their system. What represents good value for one doesn't even register for the other. So value judgments are determined by the intended end-user and defined by rival products. As to writing off our products as business expenses, you are out on your own on this one. I don't think even the most dead-eyed tax inspector would question the need for someone paid professionally to write audio reviews claiming the equipment used to write those audio reviews as a legitimate tax expense. I can see why this engenders jealousy, because I get incredibly jealous of those who can write their guitars against tax. But even in the depths of that jealousy, I recognize that those who can declare guitars as business expenses use their guitars as tools of their trade. I can't because I don't, but I do not think that they are any less entitled to claim guitars in this manner, because they use guitars in the execution of their daily work. The same applies to someone declaring a snow-plough, a kitchen, an Audio Precision analyzer or an Audio Research DAC; if they are using it for their business, it is a business expense.

Whether something is a bargain or not hangs on a lot of things. Most important, however, is how a product works in context. If a £500 loudspeaker sounds slightly better than £300 loudspeaker, not quite as good as a £700 loudspeaker, and on a par with other £500 loudspeakers, it may be a good product, but only reasonable value. On the other hand, if the £500 loudspeaker sounds as good as a £5,000 loudspeaker, that would be exceptional value for money. While I don't necessarily hold to the term, I wouldn't have a problem with someone classifying such a product as a 'bargain'. The same protocol holds whatever the price. And it's here where some of the other issues begin to take hold. Is an iPad a 'bargain'? Answer, imagining yourself as the guy saying 'do you want fries with that?'. Answer again, imagining people referring to you as 'Your Royal Highness'. So, when a car reviewer says an S-Class Mercedes is 'good value' what they mean it offers Maybach levels of luxury for less than half the price. It may not be good value to someone in the market for a Honda, but it may be considered something of a bargain for someone about to sign up for a Rolls Royce Phantom. No one says that an expensive thing is not expensive by calling it 'good value' or even a 'bargain'. It's that the expensive thing performs better than even more expensive things. None of this has anything at all to do with whether the purse-strings of the person reviewing the product. Especially as part of the accommodation thing is 'review first, purchase later.' As to 'budget', 'mid-price', 'moderately priced' or any other qualification of that sort, it is simply rendered against the entire audio market at its market price. It has to stay within the audio market; a £200 amplifier is 'budget' when the majority of other audio amplifiers cost more. I mean we could set 'budget' against the price of Corn Flakes or the price of a Troy ounce of gold, but what relevance does that have?

Let's play a little game. Imagine you and I earn the same amount of money and a similar amount of disposable income, and for both of us let's say a $10,000 system (comprising just DAC, amp and speakers) is appropriate. And let's say there's no accommodation prices You buy a $10,000 system for $10,000. I also have to buy two extra DACs, two amps and a couple of pairs of loudspeakers to have a large enough pool of products to determine where the next device under text fits in context. So that's $30,000 expenditure to review $10,000 systems. On average, that system goes through its churn cycle every three years to keep you current. Let's say you change your system after a dozen years. You spent $10,000 on your system over that time. Assuming there's no inflation, I spent $120,000, just to provide reviews for your next $10,000 system. Now, you wanna do my job without accommodation prices? Didn't think so.

The market is shrinking in unit terms. Let's look at loudspeakers specifically, in part because it represents £100m of that £200m figure. By sales volume, numbers are down across the board. By sales value, sales are static in the sub-£250 sector, down in the £250-£500 and £500-£1,000 sector and almost 50% up in the £1,000 category. The cause of this is almost entirely predicated on the iPod. This can be seen by the fact that the 2003 UK specialty audio market was worth almost £350bn prior to that, and dropped to £200bn and stayed there after 2004 (this was dubbed 'the year of the iPod' in the mainstream media). The level of 'snake oil' in the audio market was not materially changed between 2003 and 2004. If we look at markets still showing growth in traditional audio (such as China and Brazil) the bulk of the growth is in the upper sector of the market, irrespective of whether that market is one that buys high-end cables or not. Please feel free to prove me wrong, perhaps by building products you think would sell to this army of sensible buyers you claim are out there. Perhaps you are right, but more likely you'll just end up bankrupt.

The "you need to spend at least $15k on speakers" mind-set is one of the most vexatious in audio today. It has to stop. I understand the ideas behind this firewall (there's little point in someone demonstrating products that end in a 'thanks for the demonstration, but I'll save $500 by buying online'), but it is toxic for the business in general. The installation fee is another thing. There needs to be a sense of balance. In fairness, the installation of an expensive and extremely heavy set of loudspeakers can involve two or more people moving the products and a day of specialist fine-tuning that is based on a great deal specialist experience and training. It can also involve a great deal of travel to and from the dealer's store. The charges for such a service need to be higher than those for simply delivery, but possibly more in line with the service provided by a heating engineer rather than the price of hiring a lawyer. Having experienced a properly-executed professional installation, the potential level of performance upgrade is both unexpected and significant. Even to someone who's spent a great deal of time putting together systems. The importance of the install should not be under-rated. But the rate needs to be sensibly set from both parties. Years of specialist experience, knowledge and training shouldn't be devalued to the point of extinction, but neither should the bar be set so high that those who might benefit from this would never afford it.

You are absolutely correct in saying "people lie, market's don't", AudioCynic. So, where is this market of "affluent, music loving consumers"? I can only speak for the UK market (I get access to - but cannot officially publish - GfK market data statistics twice a year). The specialty audio market remained stable (at £200m) from 2004-2009 and is currently running at £190m. Areas of growth in this market include turntables, music streamers over £1,000, loudspeakers over £1,000, headphones and iPod docks. Areas that are in sharp decline are almost sector in the 'mid-price' category. This has been the case once again since 2005, and is directly attributable to the success of the iPod largely replacing that sector of the market, as well as the recent 'squeezed middle' effect of the current economic instability. A pithy way of describing the whole specialty audio market in 2011 is it's all Sonos or Sonus Faber now. There are sales at the budget end of the market, very little in between, and relative stability at the high-end (how high a high-end does vary from country to country, with £3k-£10k per unit representing the bulk of high-end sales in the UK, while $10k and above per unit forming the main thrust of US high-end sales) This might not be as directly observable by reading magazines in the US, because the magazine market is not broad as it is here in the UK. We have titles that cover (more accurately, covered) every sector of the audio market, from the über-high-end to the super-budget. Magazines go where the readers are, and if they don't they risk dying. Those that cover the low end of the industry are now concentrating on iPod docks and those that went for the middle ground have now moved upmarket or vanished. Despite this, there are products that still fill the market you describe and they do sell, but not to a 'large body of consumers' anymore. By all means, feel free to contribute some hard data on the market and how you know better. But forgive me for throwing the whole 'people lie, markets don't' right back at you.

Several issue arise from your statements: 1. "I bought the Hippo DAC at 50% off retail"... congratulations, you have just put Hippo (and the company's agents) out of business, because you stating that has set everyone reading it their discount target. "Well the price is listed at $300 but I'm not paying a penny above $160, because you sold it to the reviewer for $150!" 2. Is there the potential for a two-tier ranking of reviews, that those who loan are marked out as being a lower-class reviewer than those who buy? 3. What checks and balances are there for "I own it" fanboydom? In my experience, the worst reviewers are those who make a point of wearing their pride of ownership like a badge of honor and who tear down any product that challenges products they own because they fear they might damage their own equipment's resale value. 4. Is "I used it for three months but the review was delayed by the editor adjusting the mix of the issue and delayed again because the distributor asked if it could be put back because they didn't have enough stock in place, I returned it but it was returned to me because the distributor was out when it was delivered it a day later than the courier said and then spent the next seventeen months languishing in my paid storage until the manufacturer finally appointed a new distributor who now has a prospective client waiting for the product so I ended up holding on to the product for about two years" too much information? That's the usual story for about one in four products I review.

The main point of the exercise was to compare movement with movement, not inter-track. So yes, each test does take about half an hour to conduct, and there need to be a lot of gaps between each test, and preferably quite a few different pieces of music to compare (or a large group of test subjects). The reason for the test in this way is simple. People claim a preference of one over the other (and as you have shown the two versions are very different under close examination), and yet, if they listen to the two movements under these test conditions as complete entities, their ability to differentiate the two falls away. The standard response to this is to put it down to our poor memory for sounds, for good evidentiary reasons. What I'm interested in exploring is whether that really is the case, or if there is some other process at work. My next plan is to repeat the same whole-movement test (this time using a larger sample), in ABX, triangle and constant-reference duo/trio form. As you can imagine, given the length of the test in each case, this is not an overnight test.

2/10 for the answer 1. If the company has a demo system, that set-up is a best-case scenario for the product. It's going to be the most optimized system to highlight the performance of the device under test. Is it good because the product is good, or because of the rest of the system? You've just described a review that you would reject for lacking any frame of reference 2. Most manufacturers don't have a demo set-up. They might have a reference system that the designer uses, but that is rarely made up of new products and would not be considered relevant to the review process. Most of the other equipment they borrow (or even rent) for shows. Unless the manufacturers used get a mention as part of the review, they won't play ball. So, instead of making a more honest review schedule, the review has to roll out a series of 'thank you' statements, like a bad Oscar acceptance speech, to a bunch of companies you have no control over 3. Can a manufacturer guarantee the other manufacturers will get their part of the deal in at the same time? From experience, it's like herding cats; just trying to get a three-make system arrive within the same calendar month is something close to a minor miracle - one will be stuck with another magazine, one will be out of stock, one will arrive late because the courier left it in the warehouse overnight. We run up to six months in advance just to assume we get everything needed for a 13 review magazine in time, and we still end up scrabbling round for a couple of products in the last few weeks 4. If the magazine demands its own reference products to maintain a consistent frame of references, but was not the subject of a long-term loan, someone would have to cover the costs of a product being repeatedly sent between the reviewer and the manufacturer. Who would cover this cost? In essence, there would still be a long-term loan, just the storage of the loan product would be at the manufacturer So... sorry. Nice try. Doesn't work.

I can't afford vintage wine. I'm too busy spending money on audio equipment just to try to stay in the game.

You are the editor of a high-end audio magazine. Your publisher has just placed a blanket ban on long-term loans and accommodation purchases. Around the same time, you are offered the 'scoop' on the first review of an important, but very expensive, high-end product. The review is contingent upon the product being placed in a system of a similarly high grade. The object of the exercise is not to get fired for ignoring the publisher's edict, while also not ending up printing a magazine with a 'scoop'-shaped hole in the contents list. How would you do it? What would you do to give the public what it wants, without doing something you would consider getting your hands dirty? Sorry, but saying "I don't know, but I don't like it" doesn't count. I don't like the fact that beetroot exists in the world - it should be extinct, IMO - but that won't make the thing go away. If you think "that's your job", well... now's your chance to show how you would do it better. If you are going to simply pontificate about the evils of long-term loans and accommodation prices, but offer no solution to the problem, you are part of the problem. Put simply, put up or shut up! I'll check back in a day or so and see how you got on. I'll then provide some comments about your ideas and plans.

...is used when the reviewer thinks a product at a given sticker price performs like one costing considerably more. The reviewer can do this if they know the product and its peer group. If a reviewer thinks this product is better than others in its peer group, this should be mentioned. A bad reviewer can only judge the value of a product by the size of their own bank balance. If you think something is good value because it's affordable to you as a reviewer, fine, but you also have to determine the value of products that might cost a trivial amount to you, or that cost more money than you earn in a decade. If you cannot do that, you shouldn't be a reviewer. Personally, I don't use the word 'bargain' in audio, but not for reasons you might expect. I find the term cheap. Bargains are the discount frozen peas sold two days before they are thrown in the dumpster. Bargains are the cheap, uncomfortable couch that lasts a month longer than its warranty. Audio is much better than that.

@AudioCynic: The point of ABXing whole tracks is in the real world we don't tend to jump from orchestra to orchestra mid-note. This might lead precisely nowhere, but there is nothing in the ABX test protocol that precludes the concept of playing whole passages of music. I hold that this kind of test is more musically-directed, which is the goal of audio after all. @thesurfingalien: It doesn't sound like much, but the more people do a test like this, the closer we get to something pointing to what's really going on. Whatever that is. And the wider the sample of music used, the better too. Of course, if you cannot spot the difference between Debussy and Daft Punk, I'd call you a bit of an outlier. @wakibaki: If you want to test what I've stated, test it. If enough test what I've said and find I'm wrong, I'll happily reverse my position on this in print. If you genuinely espouse blind testing, you should test, and test, and test. This might be a reversal of the usual positions here, but if you are rejecting my statement because you 'believe' it's wrong - or because of your 'opinion' about my job – that's not really scientifically relevant. @wgscott: you blur the lines between ABX and DBT. My issue is not with DBT. If a DBT returns something different to your expectations, your expectations were probably out of line. If DBT returns something different to what's really happening, your DBT methodology is probably out of line. Other DBT methodologies are available. But, thus far, I've not been able to find any explanation why ABX is audio's DBT of choice, what tests have been used to establish its position of dominance in audio and why other, equally scientifically valid forms of DBT are rejected in favor of ABX. @chris - my reasons for getting involved in this thread is to find some form of rapprochement between the audio communities. As it stands, these petty squabbles between different groups, both militantly committed to the correctness of their argument, is merely causing more harm than good. @everyone: Perhaps a more interesting question is: What do you hope to gain from this? My concern is there is an awful lot of tilting at windmills and shots across the bows from both sides of the debate, and not a lot of actual debate. I suspect this is how it will always be and there is no chance of even the slightest rapprochement; that this is one long keyboard warrior bar-room fight. But if everything was resolved, and resolved your way... then what? What do you think would happen to the audio world? Would an audio landscape that adhered to strict objective terms be egalitarian and price-led, or dreary and totalitarian? Would a subjective audio map be a place of sonic wonder and unparalleled performance, or a unruly badlands filled with designers folly? What would you do if everything was resolved? Would you be magnanimous or spiteful in victory? Would you be philanthropic in explaining to those who didn't understand why things went your way, or would you gloat vindictively? Would you move from audio to fight another battle? For me, what I stand to gain is a more unified audio world, where it's possible to discuss concepts unthinkable to some (such as "perhaps it doesn't make a difference") and concepts unfathomable to others (such as "perhaps it does") without feeling like you have just pulled the pin on a live hand grenade. And while we are at it, perhaps be able to make a statement like "it's my money and I will spend as much or as little of it as I choose to spend, and however I choose to spend that is down to me" without being branded an idiot.

...is very difficult. You could potentially do it with Replay Gain, but then run the risk of changing then dynamic range of one or both tracks thereby invalidating the test. My suggestion is to keep the volume level identical, ensuring first that neither recording pushes the system into clipping. To do this with statistical accuracy, your wife will end up having to listen to the same movement at least 36 and preferably 48, times. It might be a good idea to find a range of music that she knows and likes where she has a couple of different versions. And buy her lots of dinners.

The thing about accommodation prices is they are universally available and linked to the trade price. If they went away, what we like or dislike wouldn't change, just what we would potentially use in our reference systems might be harder to attain. Ultimately, without accommodation, the group that suffers the most would be the readers who want us to reference the good stuff against other good stuff. A way of thinking this through is to imagine a shoe store where everything is reduced 25%. This might mean that you will be more likely to get those Bass Weejuns you always wanted instead of the no-name loafers you were intending to buy. It doesn't mean you'll walk out with a pair of outrageous bling super-expense-a-thon monstrosities that are totally incompatible with your tastes. Should accommodation be more transparent? Yes, I think it should. Will it ever be more transparent? Not until the end-results of such disclosure isn't automatically used as grapeshot by anyone with an agenda. There is no positive outcome of greater transparency at this time. If I say "I bought it at trade price" that will not appease those seeking full disclosure. Only "I bought it at trade price, which is $100 and here's the invoice and receipt for payment" will do that. At that point, you have just established the trade price as the unattainable discount; no dealer will sell it for $100 if they paid $100 for it, and no one will buy it unless they can buy it for almost exactly $100. If a reviewer cannot even afford accommodation prices and has to loan his or her system, should their reviews be reduced to a lower order of importance? Personally, I would rather give the right product to the right person than to the person with the right-sized wallet. Now, what has this to do with ABX testing?