FLAC vs WAV Part 2 Final Results
by, 02-18-2012 at 05:59 PM (21114 Views)
In part 1, I used a null test technique to show that both FLAC and WAV (lossless) file formats are identical. In this post, I have expanded the null test to cover off playing the same FLAC and WAV files dynamically from JRiver and capturing the audio waveform after the Digital to Analog conversion and analog line output stage. Here is a high level block diagram of my test setup:
For playback, I am using the exact same original FLAC and converted (by JRiver) WAV file I used in Part 1. It is Tom Petty and Heartbreakers Refugee at 24/96. JRiver is set up for bit perfect playback with no DSP, resampling, or anything else in the signal chain. I used the native Lynx ASIO driver to communicate between the sound card and JRiver. All sample rates for the tests are at 24/96.
My Win7 64 Bit HTPC build is nothing special. No special power supply or SSD or interconnects.
Side note, for Windows users, always invaluable to check your PC for latency with http://www.thesycon.de/deu/latency_check.shtml
I have tested the frequency response of my Lynx L22 sound card using REW http://www.hometheatershack.com/roomeq/ and noise levels, distortion, etc., using RightMark Audio Analyzer http://audio.rightmark.org/index_new.shtml
For capturing (i.e. recording) the audio waveforms, I used a Dell M4600 latptop and the onboard HD audio chip and driver. Here is the noise measurement of the on board sound chip. Not as good as my Lynx card above, but a check to see that everything is in working order.
I used Audio Diffmaker http://www.libinst.com/Audio%20DiffMaker.htm for recording the waveforms that were coming off the analog outputs of my playback PC. Here is the process used by Audio DiffMaker:
As an aside, I should point out that you can use this software to objectively measure anything in your audio playback chain that you have changed. Whether that be power supply, DAC, interconnects, music players, SSD, VST plugins, or whatever.
Remember, if you are audibly hearing a difference when you change something in your audio system (ABX testing), the audio waveform must have changed, and if it has changed, it can be objectively measured. I find there is a direct correlation between what I hear and what I measure. For me, to form any valid opinion about audio reproduction, I want to correlate my subjective results with my objective results and vice versa. I want a balanced view.
In the Audio DiffMaker help file, the software program is able to line up the waveforms if the program material is within 1 second of each other (protip).
Here I am capturing the first 40 seconds of TP’s Refugee in Audio DiffMaker:
I did this twice, once playing the FLAC and then the WAV, without making any changes on either computer.
To test the DiffMaker software (and everything else) is working correctly, I took the FLAC recording and compared it to itself. Theoretically, it should null itself out completely.
And it does. Ok so now let’s compare the two recordings, one FLAC and the other WAV:
What the result is saying is that the difference signal is almost -90 dB. I repeated the test ten times and obtained the same results.
You can listen to the difference track for yourself as it is attached to this post. PLEASE BE CAREFUL as you will need to turn up the volume (likely to max) to hear anything. I suggest doing this in volume level stages so you can verify there are no other artificats while listening.
As you can hear for yourself, a faint ghost track of the music, that nulls itself out completely halfway through the track and slowly drifts back into being barely audible at the end.
According to the DiffMaker documentation, this is sample rate drift and there is a checkbox in the settings to compensate for this drift:
“Any test in which the signal rate (such as clock speed for a digital source, or tape speed or turntable speed for an analog source) is not constant can result in a large and audible residual level in the Difference track. This is usually heard as a weak version of the Reference track that is present over only a portion of the Difference track, normally dropping into silence midway through the track, then becoming perceptible again toward the end. When severe, it can sound like a "flanging" effect in the high frequencies over the length of the track. For this reason, it is best to allow DiffMaker to compensate for sample rate drift. The default setting is to allow this compensation, with an accuracy level of "4".”
Of course this makes sense given that I used a different computer to record on versus the playback computer and I did not have the two sample rate clocks synched together. The DiffMaker software recommends this approach, but I have no way of synching the sample rate clock on the Dell to my Lynx card.
So when this is not possible, the DiffMaker documentation indicates to use the sample rate compensation.
However, when I tried the sample rate compensation, the DiffMaker program thru the following error:
I sent an email to the software manufacture and will follow up once I hear back.
Given that the signal is almost -90 dB from the reference and that the noise level of my Dell sound card is -86 dB, we are definitely nearing the limits of my gear. Also, given that the dynamic range of most music material we listen to is less than 20dB http://en.wikipedia.org/wiki/Dynamic_range#Audio it seems unlikely that I could hear the difference track, relative to the reference level – that’s a 90 dB difference.
Subjective Listening Tests
In JRiver, I played the FLAC and WAV (and vice versa) several times through headphones and speakers. I did this sighted and blind. I also played back the recorded reference and compare files in Audio DiffMaker using headphones. Finally, I played back the Reference + Difference track.
In my subjective listening tests, I could not hear any differences between the FLAC and WAV files in any combination of the above. Not only from the playback machine but also the recorded tracks. They all sounded identical to me. There seems to be good correlation between objective and subjective results.
As a side note, I have been into audio and music for over 40 years. For 8 of those years I was a recording/mixing engineer where I was trained and relied upon to note very small audible changes. http://www.thepikes.com/bio The reason I am saying this is because of psychoacoustic http://en.wikipedia.org/wiki/Psychoacoustics effects, our ears can be easily fooled http://en.wikipedia.org/wiki/Auditory_illusion or put in a positive way, our ears adapt to changes very quickly.
In fact, most recording, mixing, and mastering engineers use these psychoacoustic effects on purpose. For example, the HAAS effect http://en.wikipedia.org/wiki/Haas_ef...s_and_findings to make the sound more full, wider, sense of air, etc. All tricks played on our ears: http://www.algorithmix.com/en/kstereo.htm including some remastered material we download from HDTracks.
So do we not trust our ears? I am not saying that. What I am doing is bringing a balance of both subjective and objective thoughts together so we can correlate what we hear with what we measure and vice versa. Again, when performing ABX listening tests, if you are hearing an audible difference, then the waveform must have changed. If the waveform has changed then we can measure the difference.
Btw, all of the software used in these tests is free. I would encourage you to download the software’s and try this out for yourself as it does not require any special equipment. Further, you can objectively quantify any differences throughout the audio chain in your playback system.
In conclusion, using my ears and measurement software, on my system, I cannot hear or (significantly) measure any difference between FLAC and WAV. Not only just file formats, but the rest of the audio playback chain as well.