Comparing software players

[ravi1stop]

I have for some time been using a PC with Media center and Foobar 2000 as my source for audio. Recently I acquired a Mac and tested a number of software programs. During this process I became quite intrigued with finding an objective way of determining if there was any differences in the audio generated by these players or if the perceived differences were more placebo effects. I designed a test to make this as objective as possible and found there to be quite noticeable differences. There are quite a few charts involved that cannot be included in this forum but the results can be seen on a blog I created to research this subject further.Visit Audioignite high end audio blog for details. I am very interested in hearing if others have conducted similar tests.

[exa]

Very interesting results, adding objectivity to the subjective world of high end audio is a noble cause.

 

Would you be interested to repeat the experiment using J River Media Center and Foobar2000 in ASIO output mode? Can you use ASIO on the recording end as well? It will be interesting to see if the use of a different interface and different drivers will influence the results. Are we going to see more differences or less differences between the players?

[One and a half]

I have for some time been using a PC with Media center and Foobar 2000 as my source for audio. Recently I acquired a Mac and tested a number of software programs. During this process I became quite intrigued with finding an objective way of determining if there was any differences in the audio generated by these players or if the perceived differences were more placebo effects. I designed a test to make this as objective as possible and found there to be quite noticeable differences. There are quite a few charts involved that cannot be included in this forum but the results can be seen on a blog I created to research this subject further.Visit Audioignite high end audio blog for details. I am very interested in hearing if others have conducted similar tests.

 

Very comprehensive testing, and a great effort! Did you by chance do a null test of the original file and compare the different players? Mitchco did a null test with differences occuring at a very low level, Flac vs Wav I think it was, although your efforts for different players is very thoughtful.

With player software, it keeps on changing, even weekly updates, so trying to choose a player over another is not that easy. At least you were able to discern that there is a difference with measurements and to add subjective comments to reinforce what you experienced.

[mitchco]

I am very interested in hearing if others have conducted similar tests.

 

Hi ravi1stop, nice work! Maybe a bit more explanation on the charts. For example, what are the 3 axis and their scales?

 

As One and a half pointed out, I have done similar tests using null or difference testing:

 

JRiver versus JPlay

You also might be interested in Evaluating AD/DA loops by means of Audio Diffmaker

 

Keep up the great work.

 

Cheers, Mitch

[Boris75]

This is very interesting. Many thanks for the measurements. This is a very convincing way of comparing players.

 

I have been wedded to Audirvana for some time as I felt it sounded better. The charts suggest that it sounds better because it adds something to the signal, which is not what I was after. Maybe I'll test and move to Fidelia.

[John]

Great post. This seems to be an approach that could be extended. Would you be willing to compare some file formats; e.g. convert your wav file and run the test for wav vs flac vs aiff vs alac (even vs a lossy format, 320k mp3 256k aac etc...). This could shine a light on another of the ongoing debates. Thanks, John.

 

I have for some time been using a PC with Media center and Foobar 2000 as my source for audio. Recently I acquired a Mac and tested a number of software programs. During this process I became quite intrigued with finding an objective way of determining if there was any differences in the audio generated by these players or if the perceived differences were more placebo effects. I designed a test to make this as objective as possible and found there to be quite noticeable differences. There are quite a few charts involved that cannot be included in this forum but the results can be seen on a blog I created to research this subject further.Visit Audioignite high end audio blog for details. I am very interested in hearing if others have conducted similar tests.

[REShaman]

I have for some time been using a PC with Media center and Foobar 2000 as my source for audio. Recently I acquired a Mac and tested a number of software programs. During this process I became quite intrigued with finding an objective way of determining if there was any differences in the audio generated by these players or if the perceived differences were more placebo effects. I designed a test to make this as objective as possible and found there to be quite noticeable differences. There are quite a few charts involved that cannot be included in this forum but the results can be seen on a blog I created to research this subject further.Visit Audioignite high end audio blog for details. I am very interested in hearing if others have conducted similar tests.

 

I appreciate your efforts. Admittedly, I do not fully understand the significance of your charts. Instead I use my ears as a subjective tool for making comparisons, perhaps reducing the nuances of information you present, but for my purposes induce me to choose one player over another. Again a subjective decision. I have noticed that the same AIFF computer audio file can be software dependent for SQ, where one player renders the file, in my opinion, preferable compared to another. But this does not devalue the objectives of your investigations and comparisons. I reduce it to a simplistic conclusion (perspective) I have expressed as my own chart: The music rules and the equipment (include software) seduces.

Thank you for your comprehensive presentation. Enjoying the music is my ultimate goal and your research has added to the objective perspectives which are always interesting to review.

Best,

Richard

[Jud]

This is very interesting. Many thanks for the measurements. This is a very convincing way of comparing players.

 

I have been wedded to Audirvana for some time as I felt it sounded better. The charts suggest that it sounds better because it adds something to the signal, which is not what I was after. Maybe I'll test and move to Fidelia.

 

The issue in my view is, adds something versus what? ravi1stop, would be interested in more detail regarding the hardware/software chain for the "original" you used for comparison to the players.

[REShaman]

The issue in my view is, adds something versus what? ravi1stop, would be interested in more detail regarding the hardware/software chain for the "original" you used for comparison to the players.

 

Jud: Your query occurred to me as well, but I left the matter unaddressed. Posing the question opens the research to a deeper understanding of its significance.

Best,

Richard

[Boris75]

The issue in my view is, adds something versus what? ravi1stop, would be interested in more detail regarding the hardware/software chain for the "original" you used for comparison to the players.

 

It is not for me to respond in lieu of ravi1stop, but my understanding of his/her experiment is that he/she compared some signal properties of the re-encoded audio signal with the original LPCM file (which I guess may be a WAV file but the container does not matter).

[Paul R]

I am wondering (especially in light of recent conversations with Peter St.) if A+ is just leaving something in that everything else is taking out. In other words, there is a possibility that A+ is doing something more right than the others.

 

Interesting testing, and lots of interesting thinking.

 

 

-Paul

 

 

The issue in my view is, adds something versus what? ravi1stop, would be interested in more detail regarding the hardware/software chain for the "original" you used for comparison to the players.

[Boris75]

I am wondering (especially in light of recent conversations with Peter St.) if A+ is just leaving something in that everything else is taking out. In other words, there is a possibility that A+ is doing something more right than the others.

 

My understanding of ravi1stop's test is that he compares a digitized version of the analog output after using a particular player to the original digital file. So ideally, with perfect playback, there should be no difference. No difference at all is impossible with a DAD loop, but what struck me is that the charts (which I guess show energy by frequency for rolling time windows?) is that Fidelia, and to a lesser extent Pure Music, generate very little difference from the original file whereas Audirvana generates a lot. By the way, it would be interesting to see what would be the result with iTunes.

 

PS: it would be nice to see much more testing of this nature when evaluating DACs, amps, cables etc.

[SteveS1]

 

I have been wedded to Audirvana for some time as I felt it sounded better. The charts suggest that it sounds better because it adds something to the signal, which is not what I was after. Maybe I'll test and move to Fidelia.

 

Are you seriously saying that you would stop using a player you preferred, to move to a player you didn't think sounded better? I'm all for finding out why things do what they do but.....I thought the quest was for the most enjoyable replay of music. Not aligning a set of measurements. I can't begin to think of how various CDPs might have arrived at their performance, but I sure as Hell didn't pick the one that sounded worst. Funny old world now that we can analyse much more isn't it?

[Matias]

ravi1stop, great test! Thanks.

 

Could you please describe which version numbers of each player you used in the tests?

[Jud]

My understanding of ravi1stop's test is that he compares a digitized version of the analog output after using a particular player to the original digital file. So ideally, with perfect playback, there should be no difference. No difference at all is impossible with a DAD loop, but what struck me is that the charts (which I guess show energy by frequency for rolling time windows?) is that Fidelia, and to a lesser extent Pure Music, generate very little difference from the original file whereas Audirvana generates a lot. By the way, it would be interesting to see what would be the result with iTunes.

 

PS: it would be nice to see much more testing of this nature when evaluating DACs, amps, cables etc.

 

Hi, Boris. What I am wanting to know is the hardware/software detail regarding how "the original digital file" is obtained, and how the analog output of the players is digitized. My general impression is that the original file is obtained by having Pro Tools transpose (from a file? from a CD?) to disc. Then each of the software players sends a stream to a DAC, and the DAC's analog output is then digitized (using what?) to get another digital file to compare to the one on disc. There are lots of possible sources of variation from the original file along this chain, including the DAC and its internal upsampling and filters, jitter effects, etc. Whether the software players where greater variation of the digitized output from the original file is shown are responsible for it themselves, or whether they are being more accurate and thus showing more of the DAC's deviation from linearity, is something I can't say at this time based on the information presented in ravi1stop's article. (This is not at all a criticism of his fine article, it is simply saying what information hasn't yet been provided, if indeed there is a reliable way to obtain it.)

 

Re SteveS1's question, yes, it's counterintuitive, but the simple answer (and the reason I listen to a variety of music with a component to try to see if it has a "sound" of its own) is that I want my chain to be accurate rather than inaccurate but euphonic.

[jackfish]

No iTunes with BitPerfect? Might surprise us.

[SteveS1]

Re SteveS1's question, yes, it's counterintuitive, but the simple answer (and the reason I listen to a variety of music with a component to try to see if it has a "sound" of its own) is that I want my chain to be accurate rather than inaccurate but euphonic.

 

Does that include your room? My guess would be that unless you have taken some pretty drastic measures, the minuscule differences we are concerned with here will be as nothing compared to the coloration your environment contributes. I'm after engagement and involvement with my music and tend to judge any change as closer to my impression of how it should be represented or not. I have no other reference.

 

Interesting stuff.

[GoldenEar]

Very interesting comparisons! It might be interesting to compare iTunes original as a baseline with the software players, as I assume that your test file is 44kHz. I always preferred the sound of Pure Music 1.86 over the competitors, although I prefer Audirvana plus for robustness.

 

Regards

Kay

[ravi1stop]

Very interesting results, adding objectivity to the subjective world of high end audio is a noble cause.

 

Would you be interested to repeat the experiment using J River Media Center and Foobar2000 in ASIO output mode? Can you use ASIO on the recording end as well? It will be interesting to see if the use of a different interface and different drivers will influence the results. Are we going to see more differences or less differences between the players?

 

Actually the tests on Media center and Foobar were performed using Asio. I did one test going direct and the charts show very little difference between them.

[ravi1stop]

Hi ravi1stop, nice work! Maybe a bit more explanation on the charts. For example, what are the 3 axis and their scales?

 

As One and a half pointed out, I have done similar tests using null or difference testing:

 

JRiver versus JPlay

You also might be interested in Evaluating AD/DA loops by means of Audio Diffmaker

 

Keep up the great work.

 

Cheers, Mitch

 

Hi Mitch,

Very interesting way of essentially doing what I am attempting to do but with sound rather than charts. I am not familiar with Audio diffmaker but could i use it to compare the different recordings to the original?

Ravi

[ravi1stop]

This is very interesting. Many thanks for the measurements. This is a very convincing way of comparing players.

 

I have been wedded to Audirvana for some time as I felt it sounded better. The charts suggest that it sounds better because it adds something to the signal, which is not what I was after. Maybe I'll test and move to Fidelia.

 

Please do not take my work as gospel as yet as I am learning as I go and am talking with the folks at Audirvana to ensure that some setting in my system is not causing this difference. I want to make it clear that I am not saying that Audirvana is not bit perfect, just that my testing so far shows some inconsistencies that I cannot as yet explain and is hopefully some setting that I am missing.

Ravi

[ravi1stop]

My understanding of ravi1stop's test is that he compares a digitized version of the analog output after using a particular player to the original digital file. So ideally, with perfect playback, there should be no difference. No difference at all is impossible with a DAD loop, but what struck me is that the charts (which I guess show energy by frequency for rolling time windows?) is that Fidelia, and to a lesser extent Pure Music, generate very little difference from the original file whereas Audirvana generates a lot. By the way, it would be interesting to see what would be the result with iTunes.

 

PS: it would be nice to see much more testing of this nature when evaluating DACs, amps, cables etc.

 

It is a good idea to test Itunes. I will try and do that this weekend. The Weiss tests indicated very little difference between the Weiss and using the Halide bridge but there may be bigger differences between other Dacs. When I have time I will compare it to the Wyred4 sound dac. Thanks

[ravi1stop]

Hi, Boris. What I am wanting to know is the hardware/software detail regarding how "the original digital file" is obtained, and how the analog output of the players is digitized. My general impression is that the original file is obtained by having Pro Tools transpose (from a file? from a CD?) to disc. Then each of the software players sends a stream to a DAC, and the DAC's analog output is then digitized (using what?) to get another digital file to compare to the one on disc. There are lots of possible sources of variation from the original file along this chain, including the DAC and its internal upsampling and filters, jitter effects, etc. Whether the software players where greater variation of the digitized output from the original file is shown are responsible for it themselves, or whether they are being more accurate and thus showing more of the DAC's deviation from linearity, is something I can't say at this time based on the information presented in ravi1stop's article. (This is not at all a criticism of his fine article, it is simply saying what information hasn't yet been provided, if indeed there is a reliable way to obtain it.)

 

Re SteveS1's question, yes, it's counterintuitive, but the simple answer (and the reason I listen to a variety of music with a component to try to see if it has a "sound" of its own) is that I want my chain to be accurate rather than inaccurate but euphonic.

 

Sorry for not explaining my process better. The original file was made up of extracts from 4 Wav audio files compiled together in Wav lab. This wav file was then played on all the different players. The output from the PC/Mac be it via the Halide bridge directly into the Digi rack or fire wire through the Weiss dac into Spdif input of the Digirack and or the XLR analog input of the Digi rack was fed into Pro Tools and recorded as a stereo interleaved Wav file. The potential variations in the process of going through the Digirack and through protools should be common to all players and in many cases is almost identical to the original file, so I think the process was relatively repeatable and consistent. Thanks

[ravi1stop]

It is listed at the top of the charts on each post.

[ravi1stop]

Does that include your room? My guess would be that unless you have taken some pretty drastic measures, the minuscule differences we are concerned with here will be as nothing compared to the coloration your environment contributes. I'm after engagement and involvement with my music and tend to judge any change as closer to my impression of how it should be represented or not. I have no other reference.

 

Interesting stuff.

The test does not include any effect from the room. The signal path is a direct connection between the playing PC via the Dac to the Digirack and into the 2nd Pc which records the signal. I have compared analog vs digital as well. I initially started using a microphone but quickly discarded it for the reasons you point out. Thanks