• JRiver Mac vs JRiver Windows Sound Quality Comparison

    I have been listening to JRiver Media Center on Windows for almost two years and have been a happy customer. JRiver on Windows is extensively reviewed by Chris.
    Now that an early release of JRiver is available on the Mac, I thought I would take the opportunity to compare the sound quality between the two JRiver music players.
    Similar to how I compared JRiver to JPlay, I am using the following test methods and tools to compare SQ:

    • Using Audacity (or any digital audio editing software) to digitally record the output from JRiver on both Mac and Windows. Then by editing and lining up the track samples, inverting one of the tracks, and mixing them together, we will see what audio signal is left over (i.e. the difference file) and whether it is subjectively audible.
    • Using Audio DiffMaker, that is purpose built software for audio differencing tests, to analyze the two recordings, which also produces a difference file that can be listened to and subjectively evaluated.
    • Using Foobar’s ABX Comparator to listen to each recorded track and determine which one sounds different or subjectively better.

    The Audacity recordings of the JRiver music players on both Mac and Windows are included in this article so people can download and subjectively listen to and objectively inspect. Given that the test software is freeware, I designed the article to follow a step by step process, so if inclined, one can repeat the test procedures and see if the results are repeatable.









    Test Configuration and Recording Process

    The Windows computer is an Intel 3.30 GHz i5-2500 quad core with 8 GB of RAM and running Windows 7 64-bit operating system. The MacBook Pro is an Intel 2.26 GHz Core 2 Duo with 8 GB of RAM and running OSX version 10.8.2. On Windows, I am using the ASIO version of Audacity and on the Mac version 2.0.3 to record the audio bitstream from JRiver. For a DAC, I am using a Lynx Hilo, which by one objective measure, rates as one of the most transparent A/D D/A converters on the market today. The Hilo has the capability to patch (sometimes called digital loopback or route) any input to any output. As confirmed on the Lynx support forum, the audio bitstream is going from JRiver, through the ASIO driver, through the USB cable, into the Hilo, and then clocked back out the Hilo, through the USB cable, through the ASIO driver, and into Audacity. I am routing the output of JRiver to input USB Play 1&2 on the Hilo and patching it to output on USB Record 1&2 which is the input to Audacity.

    Looks like this configured on Hilo’s touch screen:






    With the Hilo I can simultaneously play audio from one software application (e.g. JRiver) and record the same audio in another application (e.g. Audacity). On Windows, it looks like this:




    On the Mac, it looks like this:




    I am using Tom Petty’s song Refugee that I downloaded directly from Tom Petty’s site which is recorded at 24/96. The Producer/Engineer’s provided a note of provenance (PDF) to go with the download, so I feel reasonably comfortable that this is as close to the master as one can get: “We made the FLAC files from high-resolution uncompressed 24-bit 96K master stereo files. When we compared those files to the FLAC’s, the waveforms tested out to be virtually identical.”

    In Audacity, the only change I made was to set the project sample rate to 96 kHz and bit-depth to 24 under the Edit menu->Preferences->Quality. Dither will be discussed later.

    Note that the “bit-perfect” light is on in both versions of JRiver while I was recording, indicating that the output of the player is streaming bit-perfect audio at 24/96 to the DAC. There is nothing else in the signal path, all DSP functions were turned off, and with ASIO, any intermediate audio layers in Windows are bypassed. All levels were set at 0dBFS and I used the stock USB cable that came with the Hilo.

    Here is what the Windows recording looks in Audacity:



    Here is what the Mac recording looks like:




    I used Audacity’s Amplify effect to validate that both recordings were recorded at the same level. Note I did not apply the amplification, this is for viewing only. On my first Windows recording, I accidentally moved the JRiver internal volume control down -0.5 dB, so the levels did not match. I did not find that out until the end and had to rerecord the Windows version. With everything set at 0 dBFS on JRiver, Audacity, and the Hilo, on both PC and Mac, the recorded levels should be exact, as depicted above. Uses the Amplify window to validate the recorded levels are the same before moving on to the next step.


    The Editing Process

    I recorded the full length of Refugee on both Mac and Windows using Audacity. First I clicked on Record in Audacity and then Play in JRiver. Once the song had played, click stop in Audacity and save the project to disk. I copied the Mac Audacity project file (.aup) and data files onto my PC and opened it up with Windows Audacity version. For waveform sample comparisons, I edited both the Mac and Windows recorded versions to roughly 60 seconds and tried to ensure that I edited the start of each track at the same sample. Windows version on the left, Mac version on the right, and I have zoomed way in to see each individual sample and placing the selection tool at the same sample point for each track:




    5,760,000 discrete samples is a good enough sample size to compare waveforms. If there is an opportunity for human error, it is in editing the start of each recording so they line up at the individual sample level.

    Resizing the waveform display windows also draws the data differently and makes it hard to properly edit. It took me more than a few tries to get it right and in the end I reverted to having the two editors opened side by side, like above. Pick a reference point and count the samples to get them aligned. One sample off will show up in the test.

    Now that I have lined the samples up, I can shift select everything to the left of the cursor, and using the cut tool, remove the samples:




    Now I can enter in 5,760,000 samples in the Selection Start field and shift select to the end of the recording. Finish by clicking on the cut (scissors) tool:



    Now I have exactly 5,760 discrete samples to export to disk:




    I followed the same process for the Mac version of the recording.



    The Comparison Process – Audacity

    Now that each of the Windows and Mac recorded samples have been digitally edited to be the exact same number of samples and hopefully the same start and end points, I can use this simple procedure to compare the two recorded tracks:

    • Import copies of both files into the same Audacity project.
    • Highlight one of the tracks, and under the Effects menu, select Invert.
    • Now highlight both tracks and under the Tracks menu, select Mix and Render. What’s left will be any difference between the two sets of recorded tracks. Save to disk.


    Here are both tracks loaded into Audacity, the top one is the Windows recorded version and the bottom one is the Mac recorded version:




    Next is highlighting and inverting one of the tracks:




    Finally, choose Mix and Render from the Tracks menu:




    This is the difference:




    No difference. Ah, but you may notice something. While I inverted one track, and highlighted both tracks, I did not mix and render, I went straight to plot spectrum.
    If I mix and render, then plot spectrum, I get:




    Note the microscopic signal that is -144 dB at 48 kHz. I do have dither turned off as per Audacity’s recommendation for export. However, reading over their lengthy description, there appears to be opportunity for inaccuracies. Additionally, looking at Audacity’s bit-depth recommendations, I should have left the default recording quality at 32-bit float and not 24 bit as -144dB is the theoretical signal to noise limit for a 24-bit digital media file. In the end, it is a moot point as -144 dB is below our absolute threshold of hearing. What does this mean? The Audacity difference test indicates that any sound quality difference between JRiver Mac and JRiver Windows is inaudible. Even if the measured difference was considerably more, say -120 dB, it may be barely audible, with headphones on, with the volume at maximum, and in a very quiet environment. However, it would be completely masked at regular program levels (e.g. -3 dBFS). If one wants to try to determine his/her ability to hear masking, try Ethan Winer’s, Artifact Audibility Comparisons.

    To verify the differencing process, here is a “control” sample of following the exact same procedure as above, but comparing the same file, in this case, the Mac file to itself:



    No difference. Check.



    The Comparison Process – Audio DiffMaker

    Audio DiffMaker is purpose built software specifically designed to automate what was done manually above as can be seen in its workflow:




    Furthermore, the differencing algorithms for time alignment and amplitude matching are optimized for this type of testing. The Help file is an excellent resource as is the AES paper on the subject of difference testing, along with Power Point slides. I am not going to go into detail as the software is readily available (i.e. free). I have also used the software in a few of my blog posts on CA, which go into more detail about test setups, software usage, and tool issues to work around.

    The process is the same as Audacity’s, except all one needs to do is load the 60 second recorded tracks, click the extract button, and watch the software work for about 10 seconds:



    Rather than trying to explain what is meant by correlation depth, one can read up on that in the DiffMaker links provided earlier in the article. If I take the DiffMaker generated difference file, open it up in Audacity, and take a frequency analysis:




    It is identical to the Mix and Render version of Audacity screen shot, right down to the decimal place. Note that Audacity opens it up as 32 bit float, yet it is a signed 24 bit PCM file. From Audacity’s dither article, there could be a very small error introduced. I should note that the first 180 milliseconds of the difference file has been edited out as that is the time it takes DiffMakers algorithms to find the correlated depth and leaves it’s processing in the file.

    As before with the Audacity test, here is the control measurement of using the same file to compare to verify the process. This time I compared the Windows recorded version to itself in DiffMaker:




    As can be seen, compares perfectly to the maximum correlation depth of 300 dB that DiffMaker is capable of. Opening the control difference file in Audacity:



    No difference. Check.


    Foobar ABX Tests

    I went into this with an expectation bias knowing there are no audible differences as verified by the two previous tests. As much as I wanted to hear a difference, both the Mac and Windows versions sound identical to me. I ran several passes in the ABX Comparator, but there is no point in posting any results as I was guessing close to 100% of the time. However, here are the two 60 second recordings of JRiver Mac and JRiver Windows so anyone can compare the files both subjectively and objectively.

    JRiver Mac 60s 33MB

    JRiver Windows 60s 33MB



    Bonus Comparison

    While I had everything set up on JRiver on the Mac, I thought I would try swapping only one thing in the audio signal chain and compare that to the recording I have already made on the Mac. So I swapped out the USB cable, changed nothing else, and made another recording:




    The one of the left is a 6ft cable that came with the Hilo. The one on the right is a London Drugs special, 5 meter shielded USB cable for $29.95. According to the Lynx Hilo manual, the longest USB cable to be used is 15 feet. The one under test here is 16 feet. Using the same Audacity procedure as before, but comparing the Mac recording with another Mac recording with the only difference being the USB cable:




    No difference. Again, this is by loading the two different recordings, inverting one of the tracks, selecting both sets of tracks, and plotting the frequency analysis.
    If I apply the Mix and Render, I get exactly the same result as both the Audacity and DiffMaker versions with the microscopic -144 dB signal at 48 kHz. And if I run the same test in DiffMaker, exactly the same result as the previous DiffMaker test between the Mac and Windows versions. As to why other folks are possibly hearing a difference using different USB cables? One anecdote is from Ken Pohlmnan’s excellent book, “Principles of Digital Audio”:




    Audiophiles have sometimes reported hearing differences between different kinds of digital cables. That could be attributed to a D/A converter with a design that is inadequate to recover a uniformly stable clock from the input bitstream. But, a well-designed D/A converter with a stable clock will be immune to variations in the upstream digital signal path, as long as data values themselves are not altered.

    As an aside, if I was to recommend one book on understanding all of the facets of digital audio today, it would be Ken’s book. The first edition appeared in 1985, and now in its sixth edition, spans 28 years of industry knowledge. There is probably no-one that knows more about digital audio than Ken Pohlmann and that knowledge is captured in his book. Highly recommended if you wish to pursue a University level understanding of how digital audio works.

    Here is the 60 second recording on the Mac using the long USB cable:

    JRiver Mac USB long cable 60s 33MB




    Conclusion

    Based on three different test methods, that I repeated more than a few times, the results indicate there is no measurable or audible sound quality difference between JRiver on the Mac versus JRiver on Windows. One could argue all I did was validate what is already known, that everything is operating to specifications. In other words, bit-perfect:

    In audio this means that the digital output from the computer sound card is the same as the digital output from the stored audio file. Unaltered passthrough. The data stream (audio/video) will remain pure and untouched and be fed directly without altering it. Bit-perfect audio is often desired by audiophiles.

    As to the reasons why this is, and if interested, I recommend Ken’s, Principles of Digital Audio book. Check out the TOC. What I really like is that the Sampling Theorem is an Appendix and the rest of the 800 pages cover literally every aspect of digital audio in every industry. Digital Audio is much more than a sampling theorem. If interested, anyone can use the same (or similar) software tools and this process to validate that the results are repeatable. One could use the files supplied, or if the DAC used supports the capability to digitally playback and record independently, one could start from scratch and validate the results are repeatable. However, I would recommend making the recordings at the maximum resolution (32-bit float in Audacity) to avoid any math or accuracy discrepancies that may occur at 24-bit resolution.

    In the meantime, enjoy the same JRiver Media Center sound quality whether on PC or Mac.










    About the author


    Mitch “Mitchco” Barnett
    I love music and audio. *I grew up with music around me as my Mom was a piano player (swing) and my Dad was an audiophile (jazz). *At that time Heathkit was big and my Dad and I built several of their audio kits. *Electronics was my first career and my hobby was building speakers, amps, preamps, etc., and I still DIY today . *I also mixed live sound for a variety of bands, which led to an opportunity to work full-time in a 24 track recording studio. *Over 10 years, I recorded, mixed, and sometimes produced over 30 albums, 100 jingles, and several audio for video post productions in a number of recording studios in Western Canada. This was during a time when analog was going digital and I worked in the first 48 track all digital studio in Canada. Along the way, I partnered with some like-minded audiophile friends, and opened up an acoustic consulting and manufacturing company. *I purchased a TEF acoustics analysis computer which was a revolution in acoustic measuring as it was the first time sound could be measured in 3 dimensions. *My interest in software development drove me back to University and I have been designing and developing software ever since.*













    Comments 98 Comments
    1. firedog's Avatar
      firedog -
      Amazing job as usual. Unfortunately there are those that will never believe the result, no matter how well you set up the test.
    1. The Computer Audiophile's Avatar
      The Computer Audiophile -
      Quote Originally Posted by firedog View Post
      Amazing job as usual. Unfortunately there are those that will never believe the result, no matter how well you set up the test.
      I think that's totally fine as long as everyone continues to enjoy this terrific hobby.
    1. VandyMan's Avatar
      VandyMan -
      Very impressive.
    1. Paul.Raulerson's Avatar
      Paul.Raulerson -
      (grin)

      But what did it *sound* like when you *listened* to it?

      -Paul

      (I just could not resist. )

      Actually, I was just being a smart tail above, but it does occur to me, that what would be an even more useful comparison is say, A+ to JRMC on the Mac, then A+ to JRMC under Windows. Turning on all the recommended options of course, except for for upsampling.

      -Paul

    1. Superdad's Avatar
      Superdad -
      I don't want the comments here to devolve into the "bits-is-bits" crowd versus everyone else--mitcho and everyone endured enough of that after his null-testing between JRiver and JPlay last year.

      But I do have a question about a sonically meaningful (to me and many others) feature of a number of the players: Integer Mode.
      I am quite certain that programs such as Amarra, Audirvana, and Pure Music would all pass the same bit-perfect null-tests as performed here--without Integer Mode turned on. But turn Integer Mode on and the music becomes (at least in my system) vastly more real. And I bet a given player could even be compared to itself (Integer Mode on/off) and pass these null tests.
      So what is really going on?
      I plan to give JRiver Mac a listen, but I'll be pretty shocked if can rival a highly evolved player like Audirvana Plus 1.4.6. I wish the two could integrate the way A+ does with iTunes.
    1. One and a half's Avatar
      One and a half -
      Thank you so much Mitchco for the analysis and the eventual conclusion. I will stick with JRMC on the Mac mini, it works well, and I get to use ASIO with my DAC The current Alpha for OSX needs a few weeks (months) to match the development on Windows, and if the same bits come out, it's a no brainer decision.

      SuperDad - JRMC doesn't need to integrate with iTunes, and there are substantial advantages :
      1. No proxy files needed for FLAC, DSF/DFF, ape and other lossless no Apple formats. This is so important as the proxy files cannot be read by say Amarra, and the proxy files are a Channel D patent
      2. Ability to import and export a library without requiring a PHD
      3. Ability to transfer library from computer to another computer without a PHD
    1. Superdad's Avatar
      Superdad -
      Quote Originally Posted by One and a half View Post
      SuperDad - JRMC doesn't need to integrate with iTunes, and there are substantial advantages :
      Sorry if I was not clear. What I want is for JMRC and A+ to integrate so that I might be able to leave iTunes behind. Perhaps that won't be necessary once I hear JRMC by itself. But I don't think JRiver supports Integer Mode, and I know they did not license the iZotope SRC engine. These are features of A+ that I can not life without.
    1. audiojim's Avatar
      audiojim -
      Quote Originally Posted by Superdad View Post
      I don't want the comments here to devolve into the "bits-is-bits" crowd versus everyone else--mitcho and everyone endured enough of that after his null-testing between JRiver and JPlay last year.

      But I do have a question about a sonically meaningful (to me and many others) feature of a number of the players: Integer Mode.
      I am quite certain that programs such as Amarra, Audirvana, and Pure Music would all pass the same bit-perfect null-tests as performed here--without Integer Mode turned on. But turn Integer Mode on and the music becomes (at least in my system) vastly more real. And I bet a given player could even be compared to itself (Integer Mode on/off) and pass these null tests.
      So what is really going on?
      I plan to give JRiver Mac a listen, but I'll be pretty shocked if can rival a highly evolved player like Audirvana Plus 1.4.6. I wish the two could integrate the way A+ does with iTunes.
      When you say amarra (which in my comparison easily bests pure music, and an earlier version of A+) sounds vastly better in integr mode -- how did you do that? I have , I think, no ability on my lion Mac mini to play amarra in integer mode .
    1. mav52's Avatar
      mav52 -
      Wow, thanks Chris, that's the same conclusion I came up ( no real difference between the two except the platform providing the base for the software). . Of course I didn't borrow my cousins Rhode & Schwarz spectrum analyzer to dig into the real details.
    1. The Computer Audiophile's Avatar
      The Computer Audiophile -
      Quote Originally Posted by mav52 View Post
      Wow, thanks Chris, that's the same conclusion I came up ( no real difference between the two except the platform providing the base for the software). . Of course I didn't borrow my cousins Rhode & Schwarz spectrum analyzer to dig into the real details.
      Hi Mav - You can thank Mitch for his thorough work on this one :~)
    1. Audio_ELF's Avatar
      Audio_ELF -
      One question... I assume that the loop-back on the hilo is done using digital connection right?

      If so, I noticing you dismiss the minute difference saying...
      Note the microscopic signal that is -144 dB at 48 kHz. I do have dither turned off as per Audacity’s recommendation for export. However, reading over their lengthy description, there appears to be opportunity for inaccuracies. Additionally, looking at Audacity’s bit-depth recommendations, I should have left the default recording quality at 32-bit float and not 24 bit as -144dB is the theoretical signal to noise limit for a 24-bit digital media file. In the end, it is a moot point as -144 dB is below our absolute threshold of hearing. What does this mean? The Audacity difference test indicates that any sound quality difference between JRiver Mac and JRiver Windows is inaudible. Even if the measured difference was considerably more, say -120 dB, it may be barely audible, with headphones on, with the volume at maximum, and in a very quiet environment. However, it would be completely masked at regular program levels (e.g. -3 dBFS). If one wants to try to determine his/her ability to hear masking, try Ethan Winer’s, Artifact Audibility Comparisons.
      Could it be that even at the minute levels that these differences are at, they can affect the Digital to Analogue conversion in a way that will be audible? By removing the DAC from the system you have eliminated this.

      I'm only putting forward a hypothosis here (in an attempt to explain why people may hear differences which measure as inaudible) ... but there could be an effect similar to the "butterfly flapping its wings..."?

      Eloise
    1. Superdad's Avatar
      Superdad -
      Quote Originally Posted by audiojim View Post
      When you say amarra (which in my comparison easily bests pure music, and an earlier version of A+) sounds vastly better in integr mode -- how did you do that? I have , I think, no ability on my lion Mac mini to play amarra in integer mode .
      I don't want to hijack this thread with other topics, but I can confirm for you that Audirvana Plus (since about version 1.4; but you must audition 1.4.6 it is fantastic) does indeed support integer mode in with Mountain Lion. Not only that, but unlike Pure Music, A+ permits integer mode to be active even when the upsampling engine (iZotope, much better than Core Audio's engine) is active. Try it out!
    1. InfernoSTi's Avatar
      InfernoSTi -
      "In the meantime, enjoy the same JRiver Media Center sound quality whether on PC or Mac."

      I haven't used a Windows computer in years at home...I'm happy to have the choice of JRiver on my Mac!

      John
    1. junker's Avatar
      junker -
      Big ups! Great job and thanks for the book reference! Looking forward to JRiver for Mac when it moves out of alpha!
    1. Quest's Avatar
      Quest -
      I just have a question - if you do the same test with jriver versus any other playback software (e.g. windows media player or itunes), is there a difference?
    1. mayhem13's Avatar
      mayhem13 -
      Good luck with this one Mitch!...lol
    1. ted_b's Avatar
      ted_b -
      Mitchco, great job in defining the problem, walking us through the testing procedures, and summarizing the results. Your organizational and presentation skills are obviously very good.

      However, haven't we already determined that bit perfect playback from different players on the same platform (let alone Mac Mini vs CAPS) sound different? I mean, after all this testing, can you tell us subjectively that your Windows server playing music through JRiver sounds the same as your Mac Mini playing music through JRiver? That (and a subjective listening comparison of MAC players against JRiver MAC) is what I'd certainly like to know, from your perspective.

      My own ears have told me that my minimalist CAPS V2+ server sounds better on JRiver than any player on my MAC Mini (short of trying JRMC there). The hardware, OS and internal software variables are many....but so be it. And I assume all of them were bit perfect, as DSD, for example, can't really be DSP'd anyway.
    1. Mercman's Avatar
      Mercman -
      Good point Ted.
    1. Audio_ELF's Avatar
      Audio_ELF -
      Quote Originally Posted by Mercman View Post
      Good point Ted.
      Good point? Well the only "point" I see is that Ted feels a subjective view tramples all over the objective evidence presented.

      I'm sorry but when people post subjective opinions that tweak x improves a system; and someone says well all objective evidence points that the difference is impossible they are told they are acting impolitely. So I call Ted's post equally impolite.

      Eloise
    1. One and a half's Avatar
      One and a half -
      Quote Originally Posted by Audio_ELF View Post
      Good point? Well the only "point" I see is that Ted feels a subjective view tramples all over the objective evidence presented.

      I'm sorry but when people post subjective opinions that tweak x improves a system; and someone says well all objective evidence points that the difference is impossible they are told they are acting impolitely. So I call Ted's post equally impolite.

      Eloise
      I agree with Eloise's opinion. Mitchco's experiments are at the digital domain, the smallest signal source level, which is correct for the experiment and the results speak for themselves. You can't hear at this level, sunshine.
      What happens after the signal is processed by DACs, cables & amps is a different matter entirely, Ted's and Mercman's subjectivitity may be based on the final signal, coming from the speakers or headphones. With so many equipment combinations, that is very possible to hear the difference.