FLAC vs WAV vs MP3 vs M4A Experiment
by, 12-02-2011 at 02:07 PM (38069 Views)
I wanted to try an experiment of measuring any differences between various media file formats as described in the title. Consider this, if you are hearing a difference when you change media file formats (e.g. from FLAC to WAV), then the audio waveform must have changed, and if it is has changed, then that change can be measured. While the waveform pictures in this article are technical, it really is a case of which picture does not belong with the others.
If you are comparing media file formats, the theory is that FLAC and WAV are lossless file formats and therefore should be identical. Meaning the waveforms are identical. My plan is to use Audacity and a well known procedure to measure the waveforms and identify any differences.
But first, how can we check our gear to see if everything is working as it should? I happen to have a function (i.e. waveform) generator and dual channel oscilloscope for viewing analog waveforms on an old school CRT.
Notice on the oscilliscope, the top trace is a sine wave. The top trace is monitoring a 20 Khz sine wave coming out of the fucntion generator and going to the analog input of my Lynx L22 sound card in my PC. Lynx card flat from 15Hz to +50Khz with no phase shift. Then it is converted into digital format and routed through the mixing console and then converted from digital to analog and monitoring on the 2nd scope channel (i.e. bottom trace).
Ok let's line it up the two waveforms a bit and see how close the waveforms are after going through analog input amp --> ADC --> DAC-->Analog output stage -> scope.
I blame any discrepancies on the +20 year old scope and the much older carbon unit operating it. Would be great to have a digital storage scope.
Ok now lets record 60 seconds of that 20Khz sine wave in Audacity at 24/96 and save it as FLAC.
This is what it looks like in digital format. Mind you I have zoomed waaaayyyy in, look at the time line. Amazing that it can be reconstructed back into a perfect sine wave again. A technological marvel.
I played back the 60 second 20Khz sine wave and lined the waves on the scope again and they matched. Same procedure for a WAV conversion of the FLAC and MP3 version. I did not want to make this repetitive, so I did not add the pics, they all look the same. My purpose was to see if everything is running as it should be.
To really see if the waveforms have been altered we are going to use Audacity and a procedure to take 2 waveforms, normalize their amplitude, massively zoom in and align the two waveforms, invert one, and mix them together. If they are identical, it will "null" out the two waveforms and there will be no signal left.
Here is the step by step procedure:
1) Import copies of both the mp3 and Wav files into the same Audacity project.
2) Amplify both to the peak volume, use the amplify effect's default value on each signal separately.
3) Zoom way in on a distinct part of the waveform. Zooming on something percussive will make this part much easier, it's easier to see.
4) Use the Time-Shift Tool to line up the waveforms exactly. Keep zooming in and adjusting until you're lined up as accurately as possible, make sure you can see each sample by the time you're done.
5) Now invert one of the signals, either will work.
6) Highlight both signals and select Tracks -> Mix and Render or Project -> Quick Mix.
7) The signal you have left will be the difference between the two original files. Everything here is what you lost when you went to the mp3 format, it's mostly high frequencies and quick changes in dynamics (such as percussion).
I chose Tom Petty's song Refugee in 24/96 FLAC format downloaded from the Tom Petty site as my "master". The reason for my choice is the attached Producer/Engineer note that comes with the download that states:
"FLAC is a “lossless” format, which sounds the same as the source files it was created from. We made the FLAC files from the same high-resolution uncompressed 24-bit 96K master stereo files we used for the vinyl and Blu ray versions of Damn The Torpedoes: Deluxe Edition. When we compared those files to the FLAC’s, the waveforms tested out to be virtually identical. With the right system, you’ll be as close to being there as we can get you."
So I am getting a copy that that the waveforms are "virtually identical" to the original analog 2 track master. Great I like that. I wish all remasters were like that. Give me an uncompressed and virtually identical waveform copy of the best mix/master that you can get your hands on. As we will see, one of the files is badly compressed and is the picture that does not belong with the others.
I used JRiver MC16 to convert the FLAC to WAV and MP3 (LAME 320Kbs). I also have the M4A iTunes version that I will compare.
The top waveform is FLAC, with the left channel on top and the right channel on bottom. And the bottom two waveforms are the left and right WAV channels. These are the original files unaltered.
Here I have zoomed way in on the waveforms to see the individual samples. Now I selected one of the waveforms and inverted the signal, and then applied the Mix and Render to produce this:
As you can see there is nothing there. It is totally nulled out. Let’s see if there is a frequency spectrum:
Ok, what about a frequency analysis:
Nothing. As expected.
Let's move to our comparison of FLAC versus MP3.
Here I have imported, amplified, zoomed, and lined the two waveforms up.
Ok let's, invert one of the waveforms and mix the 2 together:
Aha! Look, there is signal. As expected as MP3 is a lossy format.
Lets look at the spectrum:
And the frequency analysis:
As you can see, mostly high frequency content. I have attached a 30 second snippet of the file so that you can download and hear the difference with your own ears.
Well, so far the experiment is going as anticipated.
So what about M4A iTunes format? Well, lets look and see how far we get, given the the iTunes version is 2 seconds longer than the original.
Again, FLAC on top and M4A on the bottom. WOW! Look at the level of compression! And there is a time difference as well. The FLAC is 3:20 and the M4A is 3:22. So if the FLAC is from the original analog master, what is this M4A version? It looks like it is also on the CD as it says 3:22. But since I don't have a copy, I can't validate that claim. The other variable is that the M4A was downloaded from iTunes and I have no way of knowing if it has been processed in some way.
What could explain the 2 second difference? Total speculation, but I wonder if it was operator error or if the mastering lab had issues with their gear (out of calibration) or maybe the 2 different analog 2 track tape machines were not speed calibrated. Regardless, the iTunes version is horribly compressed. Time for an aside...
Why I want unaltered waveforms.
As someone who used to mix bands sound live and on tape, there is a mixing technique called, riding the fader:
Starts off with an up volume snare roll, nice intro guitar solo, then drops down in volume for the 1st verse, crank volume back up during the chorus, then back down for 2nd verse, then up for the 2nd chorus, then up for the bridge and then peak the Hammond B3 and guitar solo's a bit for a taste of what's to come, then back down for 3rd verse and then pull out all the stops for the final chorus and the rock out at the end, riding the faders to max with the peak on the piercing high note on the guitar solo and then a quick ramp down fade out.
But see, that's the sound mixer doing his/her job in getting every last bit of emotional content of the bands performance, whether live or recorded.
Have another look at the FLAC and M4A waveforms. It takes no special skill to see that the two waveforms do not match or even really resemble each other. But it is the same song! Somehow on the M4A, the low amplitude has been expanded, yet the peaks are lower... Welcome to the world of (badly done) compression.
I would speculate that when TP finished the recording and mixed to death the final two track analog master, over and over again, until it was the way they wanted it. I would think that they thought this is how it was going to get mastered on CD. I.e. an unaltered waveform copy. Not able yet to confirm, but if it is the same copy on the CD as the iTunes version, that's a world of hurt. I see on Wikipedia that there is at least 3 masters available, so who the heck knows what the iTunes one is other than it is atrocious.
I would further speculate that TP has had to live with a rubbish master job since 1979 or apparently it was remastered in the early 80's and again in 2001. Whatever, it was not till 2010 that TP finally got a copy of the original 2 track analog master tape at 24/96 and as it states the waveforms from the master and the 24/96 are virtually identical. I would even go out on a limb to say that the whole reason that Damn the Torpedoes was remastered was because of the incredibly compressed mastering(s). All speculation of course on my behalf.
The point is that the waveform on the iTunes version has been altered so much as to reduce the enjoyment factor to the point where I can't listen to it. Can you imagine what the band must have felt to have heard the iTunes version. Ruined. Again, total speculation.
You do not hear the fader riding in the iTunes compressed version. So there is no build. The verse and chorus sounds the same level, as does the bridge, solos, etc. Sounds flattened and knocked some good emotional performance out of the song. To add insult to injury, the waveform is so altered that it has also altered the soundstage. All of the psychoacoustic cues that the mixing engineer has put in place have been destroyed. In the iTunes version, TP's voice sounds flat and two dimensional. In the 24/96 unaltered copy, you can really hear his voice sitting back in the mix, very 3 dimensional sounding as are the rest of the depth cues in the remaster.
It is somewhat ironic that this is the reverse situation that plagues current remasters today. In other words, TP got burnt on his master, but his remaster is a virtually an identical copy of the original analog master. Whereas, we are now getting remasters that have their waveforms altered from the original. Can you imagine a fan playing the heck out of their favorite bands records, then getting the CD versions and now has an opportunity, 20 years later or whatever, to get a special, hi-res remaster only to find that it sounds nothing like what the fan has listened to for the last 20 years. Talk about disappointment and giving hi-res a bad name.
My example is the new CCR release. I am hesitant to buy it for exactly that reason because there is no information that I can see that states what master is being used for the remaster and what was the process used to remaster this version.
To me as an audiophile, I do not think that it is too much to ask for an unaltered waveform copy of the best master tape available for any given artist/song. Tom Petty did not think it was too much to ask. Just like the record companies did not get on-line music distribution, they are not getting that, with the advent of hi-res Blu ray video, people will pay a premium for hi resolution audio. Assuming it is the real thing and unaltered.
Was it the time or era that caused bad masters? I don't think so. Consider this, The Police's Synchronicity album was released in 1983. That's 28 years ago. The song, "Murder by Numbers" has a whopping DR of 18. Stewart Copland's drums sound incredible and in the last minute of the tune, Hugh Padgham, one of my favorite rock producer/engineers, pushes the faders up on Stewart's drum kit so high it's like he is playing right in your room.
At concert volume, the kick drum punches your stomach and the snare has such a crack, your eyes involuntary blink every time Stewart hits it - awesome! Even more so for a 16/44.1 CD mastered a long time ago. Man, I would pay a pretty penny to get a unaltered waveform copy of the original master from Les Studio.
Back to work. Let’s compare FLAC vs M4A. When I amplified the waveforms, FLAC takes 2.1db to give a peak of 0db, whereas the M4A versions only requires a .2db to reach peak. That's a factor of 10. Here it is zoomed in:
So when I inverted and mixed the 2 together, I got the following result:
Well, first of all, the red is showing clipping. Note in Audacity Beta 1.3.13, under the view menu is a menu item called Show Clipping. As a side note for the folks doing the music analysis thread http://www.computeraudiophile.com/Fo...ive-Subjective , always a good idea to have this Show Clipping checked on so that you can see if any hi-res download is clipped.
I might follow up later and see if I can pitch change the M4A to match the FLAC and re-run the test, but the bottom line is that the iTunes version is massively compressed and waveform altered. Attached is a 30 second snippet of this. Clearly you can hear the echo as one track is 1% out of sync with the other.
I also have the Blu ray version of Damn the Torpedoes. I was going to compare that too, but following Chris's Blu ray ripping video, http://www.computeraudiophile.com/co...ersion-Box-Set I could not get the software to extract the streams.
The big standout to me is how much he waveform has been altered on the iTunes M4A version. And how bad it sounds. I would pay a premium price to get the best "historical" remaster of any music without altering the waveform, which means no compression or noise reduction or eq filtering of any kind. I want the virtually identical copy. I want to hear the music the way it was intended to be heard right from the mixing chair to my listening room.
For those that are hearing a difference between lossless file formats of any kind, it is not the file format. I humbly submit that the differences you are hearing are else where. Who knows where as it could be anywhere in or around the signal path. That's the issue. It is not that the files are different, it is how your system is interpreting those files that is different. The problem is diagnosing where the differences are. Or living with the difference and pick your format that sounds best to you and move on.
What is most important to me is the erosion of original performances where the original master is gone and we don't have an unaltered waveform of a "new" master. All those analog tapes are eventually going to fall apart. I hope we have unaltered waveform copies. I understand that famous paintings undergo restorations. But even then, the restorers are trying to restore the painting to its original condition.
That does not seem to be happening in the music remastering world. Those works of art are being altered, some so bad that it does not even resemble what the artist had in mind. The bands dynamics, nuances, soundstage, tone, have all been altered to the point of ruin.
Then again, if the restoration is an enhancement of the original because of modern technology (used professionally), whose to say... Little dip in eq there little bit noise reduction here, wee bit of expansion there - the ol nip and tuck. And if done really well, it should sound like an enhanced version of the original, but restoring more dynamic range, tape hiss reduction, etc.
The idea would be to restore the tape as best as the technology allows without massive compression. Instead, if massively compressed, try dialling in the reverse settings in an expander to try and restore the original waveform (a convolver can do it). It takes a trained ear in compression to be able to apply expansion. Maybe we can get some of the original performance back.
And unless you were in the mastering room and A/B ing the changes of the ol nip and tuck, you may never know what processing occurred. Unless there are other remasters, you could compare those. But If it sounds good, does it matter?
This was an experiment. Others have had similar results. Bruce Brown from Puget Sound Studios, "If I put a wav file on one track and a FLAC file on the other track in Pyramix, I can't tell them apart, sighted or blind". And the Well Tempered Computer.
If you are hearing a difference when you change X, then the waveform must be altered in order to produce that difference. Therefore it can be measured. Don't take my word for it, try it yourself.