View RSS Feed

mitchco

FLAC vs WAV vs MP3 vs M4A Experiment

Rating: 4 votes, 4.50 average.
I wanted to try an experiment of measuring any differences between various media file formats as described in the title. Consider this, if you are hearing a difference when you change media file formats (e.g. from FLAC to WAV), then the audio waveform must have changed, and if it is has changed, then that change can be measured. While the waveform pictures in this article are technical, it really is a case of which picture does not belong with the others.

If you are comparing media file formats, the theory is that FLAC and WAV are lossless file formats and therefore should be identical. Meaning the waveforms are identical. My plan is to use Audacity and a well known procedure to measure the waveforms and identify any differences.

But first, how can we check our gear to see if everything is working as it should? I happen to have a function (i.e. waveform) generator and dual channel oscilloscope for viewing analog waveforms on an old school CRT.



Notice on the oscilliscope, the top trace is a sine wave. The top trace is monitoring a 20 Khz sine wave coming out of the fucntion generator and going to the analog input of my Lynx L22 sound card in my PC. Lynx card flat from 15Hz to +50Khz with no phase shift. Then it is converted into digital format and routed through the mixing console and then converted from digital to analog and monitoring on the 2nd scope channel (i.e. bottom trace).





Ok let's line it up the two waveforms a bit and see how close the waveforms are after going through analog input amp --> ADC --> DAC-->Analog output stage -> scope.



I blame any discrepancies on the +20 year old scope and the much older carbon unit operating it. Would be great to have a digital storage scope.

Ok now lets record 60 seconds of that 20Khz sine wave in Audacity at 24/96 and save it as FLAC.



This is what it looks like in digital format. Mind you I have zoomed waaaayyyy in, look at the time line. Amazing that it can be reconstructed back into a perfect sine wave again. A technological marvel.



I played back the 60 second 20Khz sine wave and lined the waves on the scope again and they matched. Same procedure for a WAV conversion of the FLAC and MP3 version. I did not want to make this repetitive, so I did not add the pics, they all look the same. My purpose was to see if everything is running as it should be.

To really see if the waveforms have been altered we are going to use Audacity and a procedure to take 2 waveforms, normalize their amplitude, massively zoom in and align the two waveforms, invert one, and mix them together. If they are identical, it will "null" out the two waveforms and there will be no signal left.

Here is the step by step procedure:

1) Import copies of both the mp3 and Wav files into the same Audacity project.
2) Amplify both to the peak volume, use the amplify effect's default value on each signal separately.
3) Zoom way in on a distinct part of the waveform. Zooming on something percussive will make this part much easier, it's easier to see.
4) Use the Time-Shift Tool to line up the waveforms exactly. Keep zooming in and adjusting until you're lined up as accurately as possible, make sure you can see each sample by the time you're done.
5) Now invert one of the signals, either will work.
6) Highlight both signals and select Tracks -> Mix and Render or Project -> Quick Mix.
7) The signal you have left will be the difference between the two original files. Everything here is what you lost when you went to the mp3 format, it's mostly high frequencies and quick changes in dynamics (such as percussion).

I chose Tom Petty's song Refugee in 24/96 FLAC format downloaded from the Tom Petty site as my "master". The reason for my choice is the attached Producer/Engineer note that comes with the download that states:

"FLAC is a “lossless” format, which sounds the same as the source files it was created from. We made the FLAC files from the same high-resolution uncompressed 24-bit 96K master stereo files we used for the vinyl and Blu ray versions of Damn The Torpedoes: Deluxe Edition. When we compared those files to the FLAC’s, the waveforms tested out to be virtually identical. With the right system, you’ll be as close to being there as we can get you."

So I am getting a copy that that the waveforms are "virtually identical" to the original analog 2 track master. Great I like that. I wish all remasters were like that. Give me an uncompressed and virtually identical waveform copy of the best mix/master that you can get your hands on. As we will see, one of the files is badly compressed and is the picture that does not belong with the others.

I used JRiver MC16 to convert the FLAC to WAV and MP3 (LAME 320Kbs). I also have the M4A iTunes version that I will compare.



The top waveform is FLAC, with the left channel on top and the right channel on bottom. And the bottom two waveforms are the left and right WAV channels. These are the original files unaltered.



Here I have zoomed way in on the waveforms to see the individual samples. Now I selected one of the waveforms and inverted the signal, and then applied the Mix and Render to produce this:



As you can see there is nothing there. It is totally nulled out. Let’s see if there is a frequency spectrum:



Ok, what about a frequency analysis:



Nothing. As expected.

Let's move to our comparison of FLAC versus MP3.

Here I have imported, amplified, zoomed, and lined the two waveforms up.



Ok let's, invert one of the waveforms and mix the 2 together:



Aha! Look, there is signal. As expected as MP3 is a lossy format.

Lets look at the spectrum:



And the frequency analysis:



As you can see, mostly high frequency content. I have attached a 30 second snippet of the file so that you can download and hear the difference with your own ears.


Well, so far the experiment is going as anticipated.

So what about M4A iTunes format? Well, lets look and see how far we get, given the the iTunes version is 2 seconds longer than the original.



Again, FLAC on top and M4A on the bottom. WOW! Look at the level of compression! And there is a time difference as well. The FLAC is 3:20 and the M4A is 3:22. So if the FLAC is from the original analog master, what is this M4A version? It looks like it is also on the CD as it says 3:22. But since I don't have a copy, I can't validate that claim. The other variable is that the M4A was downloaded from iTunes and I have no way of knowing if it has been processed in some way.

What could explain the 2 second difference? Total speculation, but I wonder if it was operator error or if the mastering lab had issues with their gear (out of calibration) or maybe the 2 different analog 2 track tape machines were not speed calibrated. Regardless, the iTunes version is horribly compressed. Time for an aside...

Why I want unaltered waveforms.

As someone who used to mix bands sound live and on tape, there is a mixing technique called, riding the fader:


Starts off with an up volume snare roll, nice intro guitar solo, then drops down in volume for the 1st verse, crank volume back up during the chorus, then back down for 2nd verse, then up for the 2nd chorus, then up for the bridge and then peak the Hammond B3 and guitar solo's a bit for a taste of what's to come, then back down for 3rd verse and then pull out all the stops for the final chorus and the rock out at the end, riding the faders to max with the peak on the piercing high note on the guitar solo and then a quick ramp down fade out.

But see, that's the sound mixer doing his/her job in getting every last bit of emotional content of the bands performance, whether live or recorded.

Have another look at the FLAC and M4A waveforms. It takes no special skill to see that the two waveforms do not match or even really resemble each other. But it is the same song! Somehow on the M4A, the low amplitude has been expanded, yet the peaks are lower... Welcome to the world of (badly done) compression.

I would speculate that when TP finished the recording and mixed to death the final two track analog master, over and over again, until it was the way they wanted it. I would think that they thought this is how it was going to get mastered on CD. I.e. an unaltered waveform copy. Not able yet to confirm, but if it is the same copy on the CD as the iTunes version, that's a world of hurt. I see on Wikipedia that there is at least 3 masters available, so who the heck knows what the iTunes one is other than it is atrocious.

I would further speculate that TP has had to live with a rubbish master job since 1979 or apparently it was remastered in the early 80's and again in 2001. Whatever, it was not till 2010 that TP finally got a copy of the original 2 track analog master tape at 24/96 and as it states the waveforms from the master and the 24/96 are virtually identical. I would even go out on a limb to say that the whole reason that Damn the Torpedoes was remastered was because of the incredibly compressed mastering(s). All speculation of course on my behalf.

The point is that the waveform on the iTunes version has been altered so much as to reduce the enjoyment factor to the point where I can't listen to it. Can you imagine what the band must have felt to have heard the iTunes version. Ruined. Again, total speculation.

You do not hear the fader riding in the iTunes compressed version. So there is no build. The verse and chorus sounds the same level, as does the bridge, solos, etc. Sounds flattened and knocked some good emotional performance out of the song. To add insult to injury, the waveform is so altered that it has also altered the soundstage. All of the psychoacoustic cues that the mixing engineer has put in place have been destroyed. In the iTunes version, TP's voice sounds flat and two dimensional. In the 24/96 unaltered copy, you can really hear his voice sitting back in the mix, very 3 dimensional sounding as are the rest of the depth cues in the remaster.

It is somewhat ironic that this is the reverse situation that plagues current remasters today. In other words, TP got burnt on his master, but his remaster is a virtually an identical copy of the original analog master. Whereas, we are now getting remasters that have their waveforms altered from the original. Can you imagine a fan playing the heck out of their favorite bands records, then getting the CD versions and now has an opportunity, 20 years later or whatever, to get a special, hi-res remaster only to find that it sounds nothing like what the fan has listened to for the last 20 years. Talk about disappointment and giving hi-res a bad name.

My example is the new CCR release. I am hesitant to buy it for exactly that reason because there is no information that I can see that states what master is being used for the remaster and what was the process used to remaster this version.

To me as an audiophile, I do not think that it is too much to ask for an unaltered waveform copy of the best master tape available for any given artist/song. Tom Petty did not think it was too much to ask. Just like the record companies did not get on-line music distribution, they are not getting that, with the advent of hi-res Blu ray video, people will pay a premium for hi resolution audio. Assuming it is the real thing and unaltered.

Was it the time or era that caused bad masters? I don't think so. Consider this, The Police's Synchronicity album was released in 1983. That's 28 years ago. The song, "Murder by Numbers" has a whopping DR of 18. Stewart Copland's drums sound incredible and in the last minute of the tune, Hugh Padgham, one of my favorite rock producer/engineers, pushes the faders up on Stewart's drum kit so high it's like he is playing right in your room.



At concert volume, the kick drum punches your stomach and the snare has such a crack, your eyes involuntary blink every time Stewart hits it - awesome! Even more so for a 16/44.1 CD mastered a long time ago. Man, I would pay a pretty penny to get a unaltered waveform copy of the original master from Les Studio.

Back to work. Let’s compare FLAC vs M4A. When I amplified the waveforms, FLAC takes 2.1db to give a peak of 0db, whereas the M4A versions only requires a .2db to reach peak. That's a factor of 10. Here it is zoomed in:



So when I inverted and mixed the 2 together, I got the following result:



Well, first of all, the red is showing clipping. Note in Audacity Beta 1.3.13, under the view menu is a menu item called Show Clipping. As a side note for the folks doing the music analysis thread http://www.computeraudiophile.com/Fo...ive-Subjective , always a good idea to have this Show Clipping checked on so that you can see if any hi-res download is clipped.

I might follow up later and see if I can pitch change the M4A to match the FLAC and re-run the test, but the bottom line is that the iTunes version is massively compressed and waveform altered. Attached is a 30 second snippet of this. Clearly you can hear the echo as one track is 1% out of sync with the other.


I also have the Blu ray version of Damn the Torpedoes. I was going to compare that too, but following Chris's Blu ray ripping video, http://www.computeraudiophile.com/co...ersion-Box-Set I could not get the software to extract the streams.


Conclusion:


The big standout to me is how much he waveform has been altered on the iTunes M4A version. And how bad it sounds. I would pay a premium price to get the best "historical" remaster of any music without altering the waveform, which means no compression or noise reduction or eq filtering of any kind. I want the virtually identical copy. I want to hear the music the way it was intended to be heard right from the mixing chair to my listening room.


For those that are hearing a difference between lossless file formats of any kind, it is not the file format. I humbly submit that the differences you are hearing are else where. Who knows where as it could be anywhere in or around the signal path. That's the issue. It is not that the files are different, it is how your system is interpreting those files that is different. The problem is diagnosing where the differences are. Or living with the difference and pick your format that sounds best to you and move on.


What is most important to me is the erosion of original performances where the original master is gone and we don't have an unaltered waveform of a "new" master. All those analog tapes are eventually going to fall apart. I hope we have unaltered waveform copies. I understand that famous paintings undergo restorations. But even then, the restorers are trying to restore the painting to its original condition.

That does not seem to be happening in the music remastering world. Those works of art are being altered, some so bad that it does not even resemble what the artist had in mind. The bands dynamics, nuances, soundstage, tone, have all been altered to the point of ruin.


Then again, if the restoration is an enhancement of the original because of modern technology (used professionally), whose to say... Little dip in eq there little bit noise reduction here, wee bit of expansion there - the ol nip and tuck. And if done really well, it should sound like an enhanced version of the original, but restoring more dynamic range, tape hiss reduction, etc.


The idea would be to restore the tape as best as the technology allows without massive compression. Instead, if massively compressed, try dialling in the reverse settings in an expander to try and restore the original waveform (a convolver can do it). It takes a trained ear in compression to be able to apply expansion. Maybe we can get some of the original performance back.


And unless you were in the mastering room and A/B ing the changes of the ol nip and tuck, you may never know what processing occurred. Unless there are other remasters, you could compare those. But If it sounds good, does it matter?


This was an experiment. Others have had similar results. Bruce Brown from Puget Sound Studios, "If I put a wav file on one track and a FLAC file on the other track in Pyramix, I can't tell them apart, sighted or blind". And the Well Tempered Computer.

If you are hearing a difference when you change X, then the waveform must be altered in order to produce that difference. Therefore it can be measured. Don't take my word for it, try it yourself.

Happy listening!
Attached Thumbnails Attached Files

Updated 05-09-2012 at 11:20 AM by mitchco

Categories
Personal Blogs

Comments

  1. scobek's Avatar
    Wow! I'll have to reread this after another coffee.



    I have a three AAC files and several ALAC ones, and noticed both confusingly (to me) have the same .m4a file name extension. It's been years since I last checked iTunes for music, but very much doubt that track you used was lossless. But great comparison none-the-less.



    Onya,

    Bob
  2. Miska's Avatar
    In the past (almost ten years ago), I made a similar comparison using, IIRC, Manhattan Transfer's "Offbeat of avenues" song.



    I computed average spectrum for the original and compressed, and average spectrum of the difference signal. I wrote a small piece of software to do the comparison, because different codecs have different codec delays so it had to find the correct relative shift needed to compute the difference signal.



    Here are the results:

    http://www.sonarnerd.net/projects/wavcomp/



    I was planning to make also time-domain comparison of impact on transient responses, but didn't have the energy to put up the results in a nice looking way.
  3. mitchco's Avatar
    Bob and Miska for your comments.



    Bob, you are probably right, but I have no way of knowing as the music is so compressed (and the mysterious 2 sec difference) I can't compare it to anything.



    My point is if the music (i.e. waveform) is so massively compressed, it does not matter which media file format it comes in, the musical experience is ruined already (at least for me).



    Miska, as a daytime programmer, I would be interested to know more about your project. Is it open source?
  4. esldude's Avatar
    Very nice blog entry here. Thanks for going to the trouble of pictures and screenshots and such.



    Unfortunately just what I would suspect. Both in the lossless and uncompressed files totally matching, and in how badly some remastering is done. Or otherwise butchered in the chain. Always a frustration as a consumer. I sometimes wonder why? Often it would have seemed simpler to simply do nothing rather than take time and pay someone to mess it up as sometimes happens. But not being in the music industry I am sure these decisions involve many factors beyond artist intent and sound quality.



    Again very nice, informative and educational blog entry.
  5. Julf's Avatar
    It's a pity that really great work like this doesn't get much attention (judging by the scarcity of comments), while a thread about listening chairs gets almost 100 comments. I wonder if the picture of the oscilloscope and function generator at the beginning of the blog entry scared away people?
  6. Fritsveer's Avatar
    Hi Mitchco



    Enlighting piece!

    One q: can you make a m4a from the original file and compare. May be also a 16/44 version.

    I did a comparison myself ones (starting 24/92-> 24/44 and 16/44), just to compare via listening. (NB: I failed hearing the difference at that point since my digital device could only handle 16/48).

    It will give a very good insight in the difference to the original source too. I am very curious, and hope you are be able to do that.
  7. firedog's Avatar
    If you can find the MFSL CD of Synchronicity, it has DR like the original; so I guess they used the original master and didn't volume compress it. Sounds fantastic, by the way.
  8. koogook's Avatar
    Just because something is true, factual, intellectually stimulating or similar doesnt mean that people will be interested in it juf, pop music confirms this. However testing and understanding are inperitive to the reasoning of music purchases, downloads and ripping techniques, and the thread bearer has out performed the effort a hundred fold and so you need saluting, or handshaking, or whatever the digital appreciation of some kind would be.

    I think tom petty is good, i think in 5.1 hd hes even better, clearly improving sound quality production and making it available is a good thing. If you have oscillators and spectrum analyis machines you are ten to the power of 1 of cool.

    Music companies however are slow burning fat hats and care not but for the wedge in their pocket, and are not in touch with reality, technology or a good business model.
  9. Paul.Raulerson's Avatar
    Nope, nobody scared off.



    But it would probably be a bit impolite to say that comparing an ALAC from iTunes against a RIP from a CD is like comparing Apples and Oranges. That's a lot of work to poke at.



    I would, however, be quite interested to see these results with the WAV or FLAC file transcoded to ALAC.



    -Paul
  10. mftech's Avatar
    Wow ! Your test is really impressing, since last night I'm pushing your article to various forum and friends...you should push your test result to TAS...
  11. mitchco's Avatar
    Just to clarify, the FLAC VS M4A is an invalid test as the M4A comes from a different master due to the 2 second difference between the two. I used this "invalid test" as an opportunity to comment on how over compressed the iTunes M4A version was, which was the point I was making.



    As mentioned in the post, I don't have the CD version of TP's Refugee, so I cannot compare it to the M4A, which I suspect came from the same master as both have the same song time, but again 2 seconds slower than the FLAC version. That 2 second difference is a mystery.



    All lossless files formats, including ALAC, will result in the same null test as the FLAC VS WAV. You can verify this yourself by following the null test procedure described in the post.



    Re: TAS see: http://www.avguide.com/forums/comput...quality?page=1 about 3/4 the way down the page.



    I used JRiver MC 16 to perform the file conversions and my tests were performed on a middle of the road Win 7 64 bit HTPC, with a single inexpensive 7200 RPM SATA drive. You can see the specs, including the PC's latency, here: http://i1217.photobucket.com/albums/...ola/Win7PC.jpg



    I am in the process of expanding this null test approach to include the Digital to Analog Conversion and the analog line output stage right up to the preamp inputs. The way to do this is to hook up a digital storage scope like http://www.bitscope.com/software/dso/ to the analog outputs of the DAC and capture/store the audio streams for comparison.



    Now this has become a dynamic test from the music player, through the audio signal chain, to the preamp, rather than a static file format test.



    In addition to comparing the digital and analog chain, this approach can be used to compare the waveforms not only from different file formats, but from different music players as well.



    As an electronics and software guy, I believe that there is a correlation between what we hear and what we measure and vice versa. That has been my experience over 30 years doing both, 8 of which in pro audio.



    I also believe that if you are hearing a difference in an A/B comparison scenario, then the audio waveform (electric or acoustic) has to have been altered in some way. If it has been altered, then we can measure the difference.



    If folks are claiming hearing something different when comparing A to B, then their must be a measurable difference. If there is no measurable difference, then...



    Happy listening!
    Updated 05-09-2012 at 11:21 AM by mitchco
  12. mftech's Avatar
    Hi Mitchco,

    look at this tool (Hear-The-Difference) it's very useful, at once I used it to compare two different mix, I got complete silence until the change in the mix occur.

    Impressive, specially that I cranked my headphone output quite high to catch any residual noise.

    You could align the audio clip in time.



    Here's the link.

    http://www.ohl.to/about-audio/audio-softwares/hear-the-difference
  13. mitchco's Avatar
    Hi mftech,

    thanks for the link to Hear The Difference. I downloaded and it works perfect!



    Many other interesting VST plug-ins on the site as well...



    I also like the the target curve calculator: http://www.ohl.to/calculators/targetcurve.php I get good correlation to what is calculated with what is measured: http://www.computeraudiophile.com/en...iophile-system


    Anyway, the Hear The Difference software makes it easier to perform null tests, especially with what I propose below as I will need a way to line the waveforms up using the delay feature. Cheers!
    Updated 05-09-2012 at 11:22 AM by mitchco
  14. Old Listener's Avatar
    mitchco, thanks for taking the time to create this great post.



    I agree with your conclusions about the importance of file format versus the sins of re-mastering.



    I've bought several rounds of re-mastered CDs for my favorite classical music. The sins are not as fatal to the music as in rock and pop genres but you can hear the effect of a careful re-mastering from original sources.



    Bill