16x filter test results

[mansr]

Prompted by a discussion around the new Chord M Scaler boasting an interpolation filter with 1 million taps, I created a set of test files using different filters for people to compare. The source file was an excerpt from the first track, Norwegian Wood, of the album Göran Söllscher - From Yesterday to Penny Lane. This solo classical guitar piece was chosen since one of the test participants suggested that style of music. The CD quality original was resampled to 705.6 kHz (16x) using each of the filters and saved to 24-bit WAV format. The resulting files were presented in a randomised order to avoid any biasing of the listeners.

 

First some general comments:

• I honestly cannot hear a meaningful enough difference between the 12 tracks.
• I did not like 1, 3, 5, 7, 9 because they just sound soft and slightly muffled and off. 7 & 9 seemed to have better transient timing but it's not as realistic as 8/10 whereas 1, 3, 5 is just this mushy kind of sound. I also found 11 & 12 off but I was having trouble pinpointing how or why.
• Of your filters, I think 8 and 10 jumped out as most realistic.
• I prefer Blu2. Hands down. Far superior.
• I have listened to tracks 4 to 8, there are some minor differences but very hard to tell what exactly is the difference.
• I wouldn't be surprised if all the files turned out to be identical - if there are differences, they seem quite small to me.

 

I was tempted to put in a duplicate as a trap, but I opted not to.

 

Filter 1

This is the Sox 'rate' effect at the default 'high' quality setting. The resulting filter has 3309 taps. Frequency response:

 

Listener comments:

• Soft, warm but guitar pluck very imprecise, almost muffled.
• Soft smooth and a bit  muffled but, timbrally slightly off but not as bad as when I heard it on Mojo. But some notes seem slightly off with respect to the pluck of the string. Maybe that’s what I’m hearing when I say it’s “muffled.”
• Reverb, leisurely, timing good, detail very good, deep tones at 1 minute bit a little fuzzy.
• Leisurely, reverb, timing fine.
• Very straight forward with a digital sound to it - I did not like it.

 

Filter 2

This is the Sox 'rate' effect with the 'quick' cubic interpolation setting. Frequency response:

 

Listener comments:

• Resembles Mojo but maybe slightly better with better transients? not sure if more realistic timber.
• Less smooth and a little more harsh compared to 1. Transients and plucks seems slightly more dynamic but timbrally definitely off and there is something artificial about the transients and plucks.
• Quite leisurely, timing might be a bit off because of that, not quite as much reverb, tone good, 1 minute bit good - best so far I think.
• Not as leisurely or as much reverb so sounds "clearer" but may not be. Timing seems a bit off bit around 1 minute with deeper tones not as clear.
• Opposite of track 1 - too soft, loss of detail.

 

Filter 3

This is a 5702-tap filter created using the 'kaiserord' and 'fir1' functions in Octave. Frequency response:

 

Listener comments:

• Soft, warm, but guitar pluck not very precise, almost muffled, very similar to 1.
• Soft, smooth and a bit muffled like 1. But it seems have slightly more accurate transient and was timbrally more accurate than 1.
• Very leisurely, lots of reverb, tone deep, 1 minute bit not quite as good on tone but seems like I hear more detail.
• Again more leisurely more reverb, not as much as 1. Clarity good. Timing may be a bit off. Sounds good toward the end.
• Better soundstage (more depth) than track 1 and 2. A little bit to the warm side but I like it. In fact, I was always coming back to this track.

 

Filter 4

Like filter 3 but with different parameters resulting in 1279 taps. Frequency response:

 

 

Listener comments:

• Smooth when supposed to but good transients/guitar plucks? Not sure if more realistic timber.
• More of 2? Slightly smoother than 2 in the smooth notes and slightly better transients and slightly more realistic timbre.
• Leisurely, timing seems off, quite a bit of reverb, tone on 1 minute bit OK, detail seems good, low drone notes at end seem emphasized.
• More reverb but not as much body, feels quicker, timing OK, overall not as involving. 1 minute deep tones don't sound as much like they're coming from deep inside.

 

Filter 5

This is a Kaiser-windowed sinc of length 100000. Frequency response:

 

 

Listener comments:

• Still slightly muffled but transients slightly better than 1 & 3 but still feels imprecise.
• Soft, smooth and a bit muffled. Although the transients and string plucks are dynamic without sounding as off as 3 so maybe more timbrally realistic?
• Not as involving or leisurely, timing seems off, tone maybe just a touch light, doesn’t seem like as much detail, 1 minute bit tone not the best.
• More leisurely again, some more reverb, tone not as deep, not as involving - seems to go kind of quickly because of that.

 

Filter 6

Another Kaiser-windowed sinc, this one of length 10000. Frequency response:

 

 

Listener comments:

• Smooth, more delicate but transients may not be as good as 4?
• Soft passages are sounding smoother than 2/4 but the louder transients/guitar plucks may actually sound a little too harsh and slightly unrealistic compared to 4?
• Leisurely again, nice tone, but not quite as involving though tone seems OK - goes quite fast subjectively b/c not as involving. 1 minute deep tone seems light, drone tones at end seem light too.
• More reverb, more leisurely, timing good. Tone seems OK. Clarity OK.

 

Filter 7

Back to Sox, this time using 'ultra-high' quality with 4815 taps. Frequency response:

 

Listener comments:

• Soft muffled, transient better than 1 & 3 but still feels imprecise, close to 5.
• Soft, smooth and less muffled than 1, 3, 5. Transients are really good. On its own, it sounds really good. But not as dynamic as 8/10 or Blu2.
• Timing interesting, seems better but not quite there, leisurely with reverb. 1 minute deep tones seem light, tone in general not as good.
• Very leisurely, I like it - so leisurely that timing may be slightly off, but maybe I'm hearing the details of the timing for the first time - clarity and tone very good. 1 minute bit not as echoey but not too light.

 

Filter 8

The longest filter of the test set, this is a 10-million-tap Kaiser-windowed sinc. Frequency response:

 

 

Listener comments:

• Slightly soft but otherwise decent transients but timbre seems worse than 2/4/6.
• Soft and smooth and nice passages but nice dynamic transients and guitar plucks.
• Yep, like this one, timing and detail and tone all seem right, good reverb but not too much, 1 minute bit quite good, drone tones at end very good.
• Very nice timing and tone, good detail, 1 minute bit not as good as some others - pace medium.

 

Filter 9

Another variant on the approach used for 3 and 4 produces this 2754-tap filter. Frequency response:

 

 

Listener comments:

• Soft and slightly muffled but transients seem better than 1, 3, 5, 7.
• Soft and smooth but still slightly muffled. Transients sound about the same to me as 7.
• More leisurely, tone good, timing maybe a touch off b/c too leisurely, 1 minute tone not as good as 8, drone notes don’t seem as emphasized as 8.
• Remarked on this one while half listening to speakers - may be the best of the lot - good detail, tone seems good, 1 minute deep tones excellent.

 

Filter 10

The final 'kaiserord' + 'fir1' filter has 4228 taps. Frequency response:

 

 

Listener comments:

• Delicate, smooth good transients and good timber, maybe better than Mojo?
• Not as smooth as Blu2 but may be the closest? Transients and timbral seems most realistic and seems to have the most delicacy in the details. But it’s like it’s trying very hard to be Blu2 but just missed by a very small bit.
• Leisurely, timing seems good, like he’s waiting for 1 note to decay before attacking next. Tone good. 1 minute tone seems just a little off, drone notes at end emphasized, almost hammer-like.
• More leisurely again - seems like just enough reverb, like he's timing his attack of new notes to the decay of the old ones. 1 minute tone very good again.

 

Filter 11

At 1000 taps, this Kaiser-windowed sinc is the shortest in the set aside from the cubic interpolation (filter 2). Frequency response:

 

 

Listener comments:

• Still on the soft side, sounds similar to 9.
• Softer but the sound seems a bit off.
• Very leisurely, lots of reverb, timing seems off because of that. Tone nice and deep, good detail. 1 minute tones good, rich but not too echoey. Drone notes at end emphasized but seem well done.
• Seems like lots of reverb - not as involving somehow because of that, tone seems a little light, but then some low drone notes near the end seem to be emphasized.

 

Filter 12

To finish off, one last Kaiser-windowed sinc, this one with 1 million taps. Frequency response:

 

 

Listener comments:

• Delicate but precise transients but they seem softer, maybe I'm just tired?
• Something just sounds off. It seems smooth and delicate but something just doesn’t sound quite right. Not sure what though.
• Medium pace, lots of reverb, deep tone, 1 minute tone not as good, good detail at end.
• Not bad - but fairly fast, not much reverb - tone on 1 minute bit not the best, timing OK.

 

Thanks to all who took the time to listen and respond.

 

All the test files are still available at https://www.amazon.co.uk/clouddrive/share/C4cQeYDZ06myC1lYo6Qfbo69HbHtc1RHpgZObGLhv5D.

[Jud]

I did two listening sessions with two different players and came up with two frequently contradictory sets of impressions, memorialized here.   

 

An issue for me is not knowing what to listen for.  There may be aspects of these filters that experienced filter designers would recognize by ear (of course measurement will be more accurate), and I'd be curious to know what some of those aspects are, and their audible characteristics.  This ties into my lack of vocabulary to describe what I'm hearing.

[Superdad]

Solo classical guitar as the file used to judge filters?!  Sorry, but I think that is an epic fail.  

 

Vastly better would be a track with piano, acoustic bass, drums with rim hits and cymbals, some trumpet or coronet, and a bit of female vocal—all recorded together in a real acoustic space.

That’s the sort of track I use for many tests and for filter tuning.  Far more revealing with transients (which are the easiest to focus on, and where the differences are heard between filters) than some soft plucked nylon strings.

 

Let me know if you would like me to send you some tracks.

[4est]

20 minutes ago, Superdad said:

Solo classical guitar as the file used to judge filters?!  Sorry, but I think that is an epic fail.  

 

Vastly better would be a track with piano, acoustic bass, drums with rim hits and cymbals, some trumpet or coronet, and a bit of female vocal—all recorded together in a real acoustic space.

That’s the sort of track I use for many tests and for filter tuning.  Far more revealing with transients (which are the easiest to focus on, and where the differences are heard between filters) than some soft plucked nylon strings.

 

Let me know if you would like me to send you some tracks.

Interestingly, I got in trouble for saying something like that around here years ago.

[mansr]

14 minutes ago, Superdad said:

Solo classical guitar as the file used to judge filters?!  Sorry, but I think that is an epic fail

I knew that whatever I picked, someone would respond like that. For this reason, I asked @ecwl what type of music he felt would reveal the supremacy of the Chord resampler. Blame him, not me.

[4est]

2 minutes ago, mansr said:

I knew that whatever I picked, someone would respond like that. For this reason, I asked @ecwl what type of music he felt would reveal the supremacy of the Chord resampler. Blame him, not me.

No good deed goes unpunished!

[adamdea]

49 minutes ago, Jud said:

I did two listening sessions with two different players and came up with two frequently contradictory sets of impressions, memorialized here.   

 

An issue for me is not knowing what to listen for.  There may be aspects of these filters that experienced filter designers would recognize by ear (of course measurement will be more accurate), and I'd be curious to know what some of those aspects are, and their audible characteristics.  This ties into my lack of vocabulary to describe what I'm hearing.

What do you imagine that experienced filter designers might be hearing? Aside from filter 2 the rest of them are flat to past 20Khz and seem to have adequate stop band rejection. (except maybe filter 11). What is there to hear?

[ecwl]

1 hour ago, Superdad said:

Solo classical guitar as the file used to judge filters?!  Sorry, but I think that is an epic fail.  

 

Vastly better would be a track with piano, acoustic bass, drums with rim hits and cymbals, some trumpet or coronet, and a bit of female vocal—all recorded together in a real acoustic space.

That’s the sort of track I use for many tests and for filter tuning.  Far more revealing with transients (which are the easiest to focus on, and where the differences are heard between filters) than some soft plucked nylon strings.

On 8/5/2018 at 2:54 PM, ecwl said:

As you’ve noticed my comparison tracks, I find it’s easiest to hear the differences between different tap length WTA filters on Chord DACs either with specific instruments with obviously timbre and note strikes, e.g. piano music, or with specific instruments that highlights transients/timing, e.g. guitar plucks, hand clapping, finger snapping, drum strikes, cymbals. That is not to say that everything else don’t sound better with longer tap length WTA filters, it’s just that these things are easier to hear for comparison purposes. Honestly, if you record yourself playing the piano or clapping your hands or striking drums, or doing your own rendition of Cups, even that can work.

Haha. I'm guilty as charged. I left my original quote above but I'm glad @mansr spent all this time and effort creating all these tracks.

I definitely had my eyes opened thanks to @mansr as to what software upsampling can do.

I'm actually curious about a couple of things:

1) How long was the computation time to create filter 8 and filter 12 tracks? (assuming CPU only)

2) Is the sonic difference between say filter 8/12 and Blu2 due to Blu2's noise shaper? Rob Watts says the WTA million tap filter outputs to 56 bits and then he uses an 11th order noise shaper (instead of (pseudo)-Gaussian or triangular dither) to convert it back to 24 bits. Although I have to admit I don't fully understand what Rob Watts mean by 11th order noise shaper.

[mansr]

3 minutes ago, ecwl said:

I'm actually curious about a couple of things:

1) How long was the computation time to create filter 8 and filter 12 tracks? (assuming CPU only)

I ran it again and timed it. The 10-million-tap filter used 58 seconds to process the 86-second file. This was done in a rather inefficient way using Octave on a single core of an Intel Xeon W3680 CPU (3.33 GHz) from 2010. With purpose-written software, I'd expect at least a 10x speed-up.

 

3 minutes ago, ecwl said:

2) Is the sonic difference between say filter 8/12 and Blu2 due to Blu2's noise shaper? Rob Watts says the WTA million tap filter outputs to 56 bits and then he uses an 11th order noise shaper (instead of (pseudo)-Gaussian or triangular dither) to convert it back to 24 bits. Although I have to admit I don't fully understand what Rob Watts mean by 11th order noise shaper.

I don't really see the point in doing noise shaping at the 24-bit level. It's far below what any DAC can resolve anyway.

[Jud]

1 hour ago, adamdea said:

What do you imagine that experienced filter designers might be hearing? Aside from filter 2 the rest of them are flat to past 20Khz and seem to have adequate stop band rejection. (except maybe filter 11). What is there to hear?

 

Some of the various filters available with software like HQPlayer sound rather different from each other to me.  The same is the case when some of the available parameters are changed (particularly phase, at least when I am listening through speakers) in iZotope and SoX.  I would guess (and @Miska has remarked) that these differences, at least in his software, are more evident to him. 

 

I don't want to get off topic, because I think the purpose of this particular exercise may have been primarily to see whether filter length was something that stood out audibly for listeners. But I personally am interested in whether I would notice increased ringing, whether I ought to use a steeper filter to get rid of aliasing and imaging, and whether any of it makes much difference anyway when upconverting to DSD512.

[semente]

1 hour ago, Jud said:

 

Some of the various filters available with software like HQPlayer sound rather different from each other to me.  The same is the case when some of the available parameters are changed (particularly phase, at least when I am listening through speakers) in iZotope and SoX.  I would guess (and @Miska has remarked) that these differences, at least in his software, are more evident to him. 

 

I don't want to get off topic, because I think the purpose of this particular exercise may have been primarily to see whether filter length was something that stood out audibly for listeners. But I personally am interested in whether I would notice increased ringing, whether I ought to use a steeper filter to get rid of aliasing and imaging, and whether any of it makes much difference anyway when upconverting to DSD512.

 

I wonder how the HQ Player poly-sinc-xtr filter measures. I think it sounds quite different from the poly-sinc-short.

[Jud]

8 minutes ago, semente said:

 

I wonder how the HQ Player poly-sinc-xtr filter measures. I think it sounds quite different from the poly-sinc-short.

 

Have you tried that blind? (Yes, I'm serious. The only blinded tests I've tried are a couple of times listening to minimum vs. linear phase with speakers, and I did get it correct; but whether repeated trials would bear that out, I don't know.)

[semente]

18 minutes ago, Jud said:

 

Have you tried that blind? (Yes, I'm serious. The only blinded tests I've tried are a couple of times listening to minimum vs. linear phase with speakers, and I did get it correct; but whether repeated trials would bear that out, I don't know.)

 

I don't have that possibility, but think that the difference is quite marked anyway, more than linear vs. minimum are in my experience.

Have you tried lp vs. mp vs. asym?

I prefer lp and asym to mp but the asym filter is somewhat crude when compared to the poly-sinc-.

 

P.S.: I used Anouar Brahem's "The Astounding Eyes of Rita"; perhaps I was dreaming. ?

[Miska]

1 hour ago, semente said:

I wonder how the HQ Player poly-sinc-xtr filter measures. I think it sounds quite different from the poly-sinc-short.

 

[jhwalker]

10 minutes ago, Miska said:

 

 

That IS rather extreme   And my favorite, right now.

[Superdad]

11 hours ago, semente said:

I wonder how the HQ Player poly-sinc-xtr filter measures. I think it sounds quite different from the poly-sinc-short.

 

The poly-sinc-short is the filter I settled on several years ago—and still my favorite by a large margin (combined with NS4 or NS5 dither).  I tried the xtr variant a couple of times about 6 months ago, but it did not do much for me.  I’ll try it again soon.

[Panelhead]

  That is impressive response. Is the 0 - 20 Hz response as linear as it looks on the compressed scale of this graph? 

  I looked at the Border Patrol dac  review and saw the response to a 1K hz impulse sample.  Border Patrol has gotten a lot from a TDA1533 chip.

[Sam Lord]

On 8/16/2018 at 12:37 PM, Superdad said:

Solo classical guitar as the file used to judge filters?!  Sorry, but I think that is an epic fail.  

 

Vastly better would be a track with piano, acoustic bass, drums with rim hits and cymbals, some trumpet or coronet, and a bit of female vocal—all recorded together in a real acoustic space.

That’s the sort of track I use for many tests and for filter tuning.  Far more revealing with transients (which are the easiest to focus on, and where the differences are heard between filters) than some soft plucked nylon strings.

 

Let me know if you would like me to send you some tracks.

 

I agree completely.  I have argued for a couple decades that simple music doesn't enable the fine differences in codecs to be heard.  I prefer massed a capella voices or multiple string instruments recorded to two mics or one stereo mic as the best samples to use for discernment.

 

I think this is a major reason why symphonic music on CDs has been mostly disappointing compared to the same on tape or vinyl, and why chamber music blossomed on CD: less sound sources to expose clocking weakness and filter effects compared to good hi-rez digital or analog recording and playback.  High-quality ADCs and avoidance of downsampling to 16/44 until the end of mastering has alleviated but not eliminated the problem, IMO.     

[Shadorne]

@mansr

 

I may have missed it. What is the phase response for each filter? You show only the frequency response of the filters. Or are they all linear phase?

[mansr]

3 hours ago, Shadorne said:

I may have missed it. What is the phase response for each filter? You show only the frequency response of the filters. Or are they all linear phase? 

All are linear phase.

[Shadorne]

3 hours ago, mansr said:

All are linear phase.

Ok thanks. IMHO it will be nearly impossible to hear any differences on most filters. Filter 2 affects the audible band with a slow roll off - so it should be audibly different - depending on equipment. 

 

Did you look at passband ripple among the filters? Passband ripple  may be audible at a threshold of around 0.1 dB.