• My Lying Ears



    As a diehard card-carrying audiophile I am interested in all things related to this wonderful hobby. I've published articles based solely on my subjective listening experience and I've published articles detailing only objective measurements and facts about products. I enjoy publishing and reading articles that cover the gamut. I also think it's healthy and interesting to be open to perspectives completely incongruent with our own. With this in mind, I was recently sent a link to the JRiver forum to read a post about one person's perspective and experience as an inquisitive listener. I really liked what I read, in the sense that it's a real world story to which many people can probably relate and it was written in a non-confrontational way. In fact every audiophile I know, golden-eared or not, has at one time or another experienced something very similar to the follow story. I'm not pushing any agenda or endorsing a point of view by publishing this article. I simply think a worthwhile read for all who enjoy this hobby as much as I do.

    Here is a a re-written, more complete version of the post, sent to me for publication by the author Michael.


    Recently on the Jriver forums (Link) a forum regular was describing his experience at an audio shootout where three bit perfect players were compared. Jriver had not done particularly well in the tests (only receiving 4 out of 38 votes), and there was some discussion of why that might've been the case, given that all three players were (at least notionally) bit-perfect. There were some questions about the test methodology (you can see all the gory details in the linked thread), and some good discussion about how bit-perfect players might or might not conceivably sound different. Ultimately several forum members were of the opinion that the test was basically invalid, while others thought that surely, because so many people heard a difference that there must've been a real difference to be heard.

    My own view on this issue is complex. I will confess that I have occasionally heard differences between bit-perfect players. But I don't believe that bit-perfect players actually sound different. That may sound like a paradox, so I'll follow it up with a second one: I don't trust my own ears to correctly detect those kinds of differences in audio. You might well ask “Why not?” Let me offer an embarrassing personal anecdote to explain my point of view about listening tests and the fallibility of the ear:

Several years ago I built a pair of home-made bi-amped speakers. They're each the size of a large washing machine and they took me the better part of a year to build (more than a month of Sundays). Because they were entirely home-made and I was trying to do an active crossover from scratch, even after they were structurally complete, they still required quite a bit of tweaking to get the crossovers dialed in and the EQ set. 

So I started by just dialing in the EQ that seemed to make sense based on the specifications of the drivers, and taking a couple of quick RTAs with pink noise. That sounded alright, and all of my friends (several of whom are musicians and/or “sound guys”) dutifully told me how great they sounded. There was just one hitch: I kept getting headaches whenever listening to the speakers, and the headaches would go away right after I turned them off. So I tried to solve the problem by tweaking some frequencies with EQ. After some tweaks, I'd think I'd made some progress (it sounded better!), and everyone who heard the changes thought the new EQ sounded better.

    Eventually, I even started dutifully "blindly" A/Bing new EQ with the old EQ (I'd switch between them during playback without telling my guests what I was switching, which isn't really blind at all), and my guests would invariably swear the new EQ sounded better. And I kept going with this "tuning by ear" method, often reversing previous decisions, backing and forthing and adding more and more convoluted filters. 

The most embarrassing moment (and something of a turning point) was when I was A/Bing a filter, and a friend and I were convinced we were on to something really excellent. After ten minutes of this, we realized that the filter bank as a whole was disabled. I had been toggling the individual filter, but the bank of filters wasn't on, so it wasn't actually even affecting playback at all. And we had been very convinced we heard a difference. And the headaches never went away.

Eventually the headaches (and a growing skepticism) prompted me to stop screwing around and take some real log sweep measurements (at the suggestion of one my more empirically-minded friends). Once I did, I realized that there was apparently a huge (10+ dB) semi-ultrasonic resonant peak at 18.5KHz that I couldn't even actually hear. So I fixed it and verified the fix with measurements. And then my headaches went away. 

This prompted me to take an agonizing look at the rest of the measurements and noticed that my "tuning by ear" which I (and my friends) all felt was clearly superior had turned the frequency response into a staggering sawtooth. So I systematically removed the EQ that was pushing things away from "flat," and kept the EQ that contributed to flatness, and re-verified with measurements. The result sounded so different, and so much more natural that I was embarrassed to have wasted months messing around trying to use my "golden ears" to tune my speakers. And my wife (who had been encouraging, but politely non-committal about my EQ adventure) came home and asked unprompted if I had done something different with the speakers, and said they sounded much better. And she was right; they did. In a few afternoons, I had done more to move things forward than I had in months of paddling around. 


    The point of this anecdote is not to try and prove to anyone that my measurement-derived EQ sounded better than my ear-derived EQ or that a flat frequency response will sound best: as it happens, I ultimately preferred a frequency slope that isn't perfectly flat, but I couldn't even get that far by ear. 

The point is that taking actual measurements had allowed me to:


    1) Cure my ultrasonic frequency-induced headaches;

    2) Improve the fidelity of my system (in the literal sense of audio fidelity as "faithfulness to the source"); and

    3) Ultimately find the EQ curve that I liked best (which looked nothing like my ear-tuned curve).



    My ears (and the inadvertently biased ears of my friends) did not allow me to do any of those things, and in fact led me far astray on issue 2). My ears couldn't even really get me to 3) because I kept reversing myself and getting tangled up in incremental changes. Most damning, my ears were not even reliably capable of detecting no change if I thought there was a change to be heard. 

Once I realized all this, it was still surprisingly hard to admit that I had been fooling myself, and that I was so easily fooled! So I have sympathy for other people who don't want to believe that their own ears may be unreliable, and I understand why folks get mad at any suggestion that their perception may be fallible. I've been accused by many indignant audiophiles of having a tin ear, and if I could only hear what they hear, then I'd be immediately persuaded. But my problem is not that I am unpersuaded: it's that I'm too easily persuaded! I'll concede, of course, that it's possible that I have tin ears and other people's ears are much more reliable than mine, but the literature concerning the placebo effect, expectation bias, and confirmation bias in scientific studies suggests that I'm probably not entirely alone. 

And I've seen the exact same phenomenon played out with other people (often very bright people with very good ears) enough times that I find it embarrassing to watch sighted listening tests of any kind because they are so rarely conducted in a way designed to produce any meaningful information and lead into dark serpentines of false information and conclusions. 



    

So to bring things back around: if some bit perfect audio players have devised a way to improve their sound they have presumably done so through careful testing, in which case they should be able to provide measurements (whether distortion measurements on an analog output, digital loopback measurements, measurements of the data stream going to the DAC, or something) that validates that claim. If they claim that their output "sounds better" but does not actually measure better using current standards of measurement, they should be able to at least articulate a hypothetical measurement that would show their superiority. If they claim that the advantage isn't measurable, or that you should "just trust your ears" than they are either fooling themselves or you.

In a well-established field of engineering in which a great deal of research and development has been done, and in which there is a mature, thriving commercial market, one generally does not stumble blindly into mysterious gains in performance. Once upon a time you could discover penicillin by accident, or build an automobile engine at home. But you do not get to the moon, cure cancer, or improve a modern car's fuel efficiency by inexplicable accident. In an era where cheap-o motherboard DACs have better SNR's than the best studio equipment from 30 years ago, you don't improve audio performance by inexplicable accident either. If someone has engineered a "better than bit perfect" player they should be able to prove it, as they likely did their own testing as part of the design process. If they can't rigorously explain why (or haven't measured their own product!), let them at least explain what they have done in a way that is susceptible of proof and repetition. Otherwise what they are selling is not penicillin, it's patent medicine. 

Bottom line: if you and a group of other people hear a difference, there may really be a difference, but there may not be too. Measurements are the easy way to find out if there is really a difference. Once you've actually established that there is a real, measurable difference, only then does it make sense to do a properly conducted listening test to determine if that difference is audible. Otherwise you're just eating random mold to find out if it will help your cough (or headache, as the case may be).

    Or you can do what I do for the most part these days: just relax and enjoy the music.


    - Michael



    Comments 248 Comments
    1. kirkmc's Avatar
      kirkmc -
      Funny, I mentioned on the CA forum the other day something that happened to me regarding one of these apps. I write for Macworld, and some years ago I was asked to look at one of these audiophile players for a potential review. I tried it out for a while, and found no difference between this app and iTunes, at least in sound quality. I contacted the company - who will remain nameless, but let's just say it's one of the Big Names in Audiophile Music Player Apps - and had a phone call with the lead developer. In about a 30 or 40 minute conversation, he was unable to tell me exactly how his app was supposed to make the music sound better. There were some things, such as loading files into RAM, hog mode, and the like, which have (IMHO) questionable value, but aside from that, he simply couldn't tell me why his app sounded better.

      Needless to say, we passed on the review.
    1. Jud's Avatar
      Jud -
      *This* ought to set the cat amongst the pigeons.

      My own view is that both Michael and the "other side" are right. I think we can be easily fooled *and* that subjective experience can ferret out surprising amounts of valuable information. A long time ago I ran a blind test here, and promised to sum up what I thought it all meant, which I haven't yet done. What I'll say at this point about my view is that our perceptions and especially the brain's internal processing are both specific and complex, and broad generalizations are thus unlikely to be completely correct.
    1. Nicholas_S's Avatar
      Nicholas_S -
      Beauty is where you find it...
    1. DanRubin's Avatar
      DanRubin -
      Quote Originally Posted by Jud View Post
      *This* ought to set the cat amongst the pigeons.
      One of my favorite British expressions!
    1. cmarin's Avatar
      cmarin -
      Great article Chris. Thank you for sharing.

      I'd add that I've also found my ears/brain to be temporally inconsistent with respect to preferences. There have been instances when I found a change preferable at the end of a long day of listening, only to reverse the preference with a "fresh" set of ears the following day; and vice-a-versa.

      Oh well, it could be worse!
    1. dallasjustice's Avatar
      dallasjustice -
      This is great. More self-effacing anecdotes are sorely needed on audiophile forums. You know, the place where everyone has a golden ear? I also think if more folks worked with microphones and DSP/room correction they would more likely encounter strange placebo observations like the one described here.

      Last week I thought the bass was shy on a few albums. I didn't get to listen much, but I just blamed it on bad mastering/recording quality. A couple of days ago, I looked at the Lynx Hilo mixer and noticed that my subs had been reduced to -12db. I set them back to 0 and the bass was great again!
      "Things happen, that's all they ever do", Dawes.
    1. REShaman's Avatar
      REShaman -
      I concur with the author's conclusion. And have been ending my posts since I joined CA in 2011 most of the time with "enjoy the music" for that has always been my highly-valued criterion among the many. Lately with "music's the thing; the equipment seduces", given the innovations of hardware and software combinations that have been introduced most recently. I do not dispute the value of measurements for shaping an auditory outcome for designing and calibrating etc.

      However, for me, sensory specific feedback is the "measurement" I rely on and trust above all no matter what the complex of "objective" measurements reveal.

      What I hear has merit; and what I feel about what I hear is my pathway to the enjoyment of music. Differentiation in hardware and software synergies exist and Change is the true consant in the service of Truth. Ultimately, I am the best (only) instrument for the measurement of my enjoyment of music.

      Thank you, Chris for another angle of appreciation for what brings us together.

      Richard
    1. sdolezalek's Avatar
      sdolezalek -
      I wonder to what degree the lack of a "common language" holds back our ability to interpret and discuss what we like and dislike about audio products. In my own mind I try to break sound qualities into the following general buckets:

      1. Frequency response (is it flat or at least linear if tilted down and to the right?) Although easily measured, room response effects can mess up an otherwise pristine starting point, so how sure are you about what you actually hear -- maybe a nonlinear piece of equipment well matches my non-linear room and sounds horrid in yours?)

      2. Phase response and coherence Am I being negatively influenced by the arrival time of different frequencies at my ears? Most comments about good or bad imaging or sound stage probably fall into this category. Similar issue as #1 if incoherences are created by different pieces in the signal chain then the end result depends highly on equipment matching as some may cancel each other and others may reinforce the error. These are also measurable at the source, but can vary by listener position and ignores rear wave effects, particularly from dipole speakers.

      3. Distortions (does one product seem to remove a haze that hides fine detail when compared to another?) This can include jitter, analog line noises, filtering artifacts, etc. so it's a big category but it seems clear that taking these away can "improve" the sound we hear). For example the "tighter bass" reported by most users here upon inserting an Uptone Regen in their system, I would interpret as having removed some form of clouding distortion. Similarly, a lot of the upsampling/filtering discussions appear to be about moving possible noise artifacts out of the audible range.

      4. "Other" which can include stuff that is measurable and should really be its own category above (known others) and stuff that we think we hear but can't seem to either measure or define (unknown others). I would generally put comments like "more air," "smoother sound," "more analog sounding," "better prat," and "wider, deeper soundstage" into the unknown others category because we aren't really speaking to something that we can tie to a known or measurable equipment issue.

      I would think that the more often we can assign a positive or negative experience to an identified bucket rather than the "other" category, the more likely that we can (a) agree about what we are hearing, and (b) address it. Stuff that stays in the other, and especially the unknown others category is likely to remain the subject of strong and often incoherent argument...

      What other "defined" categories should be on my list above?
    1. Bystander's Avatar
      Bystander -
      I would hope that there are fewer and fewer audiophiles out there who would disagree with any of these obvious truths.
    1. Jud's Avatar
      Jud -
      Quote Originally Posted by sdolezalek View Post
      I wonder to what degree the lack of a "common language" holds back our ability to interpret and discuss what we like and dislike about audio products. In my own mind I try to break sound qualities into the following general buckets:

      1. Frequency response (is it flat or at least linear if tilted down and to the right?) Although easily measured, room response effects can mess up an otherwise pristine starting point, so how sure are you about what you actually hear -- maybe a nonlinear piece of equipment well matches my non-linear room and sounds horrid in yours?)

      2. Phase response and coherence Am I being negatively influenced by the arrival time of different frequencies at my ears? Most comments about good or bad imaging or sound stage probably fall into this category. Similar issue as #1 if incoherences are created by different pieces in the signal chain then the end result depends highly on equipment matching as some may cancel each other and others may reinforce the error. These are also measurable at the source, but can vary by listener position and ignores rear wave effects, particularly from dipole speakers.

      3. Distortions (does one product seem to remove a haze that hides fine detail when compared to another?) This can include jitter, analog line noises, filtering artifacts, etc. so it's a big category but it seems clear that taking these away can "improve" the sound we hear). For example the "tighter bass" reported by most users here upon inserting an Uptone Regen in their system, I would interpret as having removed some form of clouding distortion. Similarly, a lot of the upsampling/filtering discussions appear to be about moving possible noise artifacts out of the audible range.

      4. "Other" which can include stuff that is measurable and should really be its own category above (known others) and stuff that we think we hear but can't seem to either measure or define (unknown others). I would generally put comments like "more air," "smoother sound," "more analog sounding," "better prat," and "wider, deeper soundstage" into the unknown others category because we aren't really speaking to something that we can tie to a known or measurable equipment issue.

      I would think that the more often we can assign a positive or negative experience to an identified bucket rather than the "other" category, the more likely that we can (a) agree about what we are hearing, and (b) address it. Stuff that stays in the other, and especially the unknown others category is likely to remain the subject of strong and often incoherent argument...

      What other "defined" categories should be on my list above?
      I agree - I think this is hugely important. I wish reviewers, developers, and we listeners together would try to come up with verbal descriptions for the effects of various filter characteristics. Can we describe the sonic difference between a filter with group delay and one without? Linear phase and minimum phase? Aliasing/imaging vs. ringing? These are central to the sound of our DACs, yet we try to make do with a vocabulary developed to describe tubes and turntables.
    1. Sal1950's Avatar
      Sal1950 -
      Great article, thank you Michael. It's so good to hear the voice of reason here.

      "So to bring things back around: if some bit perfect audio players have devised a way to improve their sound they have presumably done so through careful testing, in which case they should be able to provide measurements (whether distortion measurements on an analog output, digital loopback measurements, measurements of the data stream going to the DAC, or something) that validates that claim."

      That is the bottom line. There is no "other side" to a scientific result.
    1. Paul R's Avatar
      Paul R -
      Nice story. And he is correct in what he is saying.

      Only thing, my scope doesn't enjoy the music, I do. So if my ears tell me that this or that sounds better, then I am pretty much going to go with the ears. Why be discontent with an expensive music system? Sure, you can always make it better, but the cold hard facts are pretty plain- most of what we do to "improve" the sound is either just accounting for room distortions, equipment distortions, or distortions that only occur inside our heads.

      Any of the three can ruin your enjoyment of the music.

      -Paul
    1. Audio_ELF's Avatar
      Audio_ELF -
      Quote Originally Posted by DanRubin View Post
      Quote Originally Posted by Jud View Post
      *This* ought to set the cat amongst the pigeons.
      One of my favorite British expressions!
      And a great work by Agatha Christie...

      As to the original topic ... its always been my opinion that we have to be careful to divide observations (listening "proof") over what we enjoy, and observations that we present as evidence.

      I have no problem with people saying "I enjoy listening to Music Player X more than Music Player Y" and that statement is there as their opinion and is not open to being questioned or requiring any justification. However when someone makes a statement along the lines of "I listened and Music Player X is clearly producing a better sound quality than Music Player Y" then I think that is being presented a "fact" and therefore is open to challenge over methodology used to come to that conclusion.

      As an aside, I always feel such proof should include the fact that you can tell when things having changed. For example if you are "blind" testing two software players by having someone switch back and forward and asking you if can hear a differences, sometimes when they say they have changed, they should stick with the same player. Of course so many audiophiles will say any form of blind testing is too stressful and therefore the differences will always fail to be noticed...

      Eloise

      PS. measurement can never tell you which of two products sound "better"; it is simply a starting point for confirming that two products are even different in the first place (IMO).
    1. 4est's Avatar
      4est -
      An interesting story, but I do not see how the bit perfect players relate to crossovers and EQ. In the article, the author got "lost in the weeds" and no longer had a good sonic reference point. Once things are askew, it IS hard to get back, but BP players do not do that. This could almost be read as an op ed for why JRiver lost the shootout.
    1. miguelito's Avatar
      miguelito -
      Thx Chris. I agree with you on this. However, I'd like to point out the following: I am a physicist by training (PhD in high energy experimental physics) and I believe that everything that can be heard in an audio system is measurable. The catch is you might not know what it is that you need to measure. Consider for example the MQA claim about time-correct response: the concept (talked about infinitely) of higher frequencies being subjected to a phase shift (akin to a time shift) as a source for a big difference in audio quality perception is something that is not entirely obvious. That is, even though the frequency response is "flat", the phase shift causes our brains to dislike the result. In other words, there might be subtle things that we are very sensitive to and less subtle things that don't matter as much. It might well be true that the reason higher res sounds better is because of the lack of phase shifts in the audible part of the spectrum than the actual frequency extension (obviously I am not saying anything new here). Bottomline: every true (non-placebo) effect must be measurable, it's just it might be tricky to figure out what to measure.
    1. maelob's Avatar
      maelob -
      great article - i suspect that most of the audio companies are aware of this phenomena and a lot of them use it to their advantage. however i feel some people will keep buying stuff no matter what. also we are contantly bomarded from all sides about products and the never ending tweaks. it is a real tough issue.
    1. miguelito's Avatar
      miguelito -
      Quote Originally Posted by miguelito View Post
      Thx Chris. I agree with you on this. However, I'd like to point out the following: I am a physicist by training (PhD in high energy experimental physics) and I believe that everything that can be heard in an audio system is measurable. The catch is you might not know what it is that you need to measure. Consider for example the MQA claim about time-correct response: the concept (talked about infinitely) of higher frequencies being subjected to a phase shift (akin to a time shift) as a source for a big difference in audio quality perception is something that is not entirely obvious. That is, even though the frequency response is "flat", the phase shift causes our brains to dislike the result. In other words, there might be subtle things that we are very sensitive to and less subtle things that don't matter as much. It might well be true that the reason higher res sounds better is because of the lack of phase shifts in the audible part of the spectrum than the actual frequency extension (obviously I am not saying anything new here). Bottomline: every true (non-placebo) effect must be measurable, it's just it might be tricky to figure out what to measure.
      And BTW... I want to add to this that, although I perceive differences in my system, for example with upsampling vs no upsampling, it is almost always the case that I cannot nail down the reasons for the differences (like frequency extension or bass authority or what have you). I just find I enjoy the music more with some configurations vs others.
    1. crenca's Avatar
      crenca -
      Thanks for posting this. Being new to the "audiophile" world, I was perplexed by this article when I read it last night:


      Two USB Cables | The Absolute Sound


      Specificaly, how can the physical properties/quality of a usb cable specifically target "the middle range of the musical spectrum" of a digital signal? How can the "analog" electrical pulse that travels down a USB cable be effected in such a way (and it is of course influenced by its physical environment as all physical/"analog" signals are) that specifically targets digital information only in "the middle range of the musical spectrum", and in such a way that the buffer/DAC is "influenced" by said information change and passes it on down the chain? It defies any explanation that I am aware of - and I suspect it is all in the reviewers head.


      Again, thanks for the perspective this essay provides...
    1. Jud's Avatar
      Jud -
      Quote Originally Posted by miguelito View Post
      Bottomline: every true (non-placebo) effect must be measurable, it's just it might be tricky to figure out what to measure.
      Agree completely.

      Those of us of a certain age recall when jitter was not yet a well known phenomenon outside the research community, when people who didn't like what they heard were told they were just being stubborn in refusing to accept the new format with superb measurables (vanishingly low wow and flutter!). Or before oversampling came into wide use, how they were ignoring the mathematical proof of the Sampling Theorem, and thus subjectively disliking essentially perfect reconstruction of the original sound.

      Before even figuring out what to measure, you have to be aware something is there *to* be measured.

      None of the most foregoing is to say that we aren't well capable of fooling ourselves.
    1. ecwl's Avatar
      ecwl -
      Just to make this more controversial. John Swenson has said that his Uptone Regen reduces ground plane noise and packet jitter which in turn reduces the reduces the jitter in the DAC clocks all of which are measurable but require very expensive equipments to measure. He believes these differences is what causes the audible sound improvements. What if the 3 different bit-perfect players happen to generate different amounts of ground plane noise on the USB bus... But then maybe it's all placebo...