I've had trouble getting DSD128 to work on the Mytek. Software/firmware is latest as of 08/2013 for Mytek and Win8 running JRiver v18. I've sorta found a work-around and thought I'd share as I've not found these answers in my searches, nor after reading most, but not all, of this thread.
Firewire works for DSD64 but no settings seem to work for DSD128. However, I can get USB to work (just a standard USB 2 output; no audiophile PCI). However, I must check both DSD and DoP under bit streaming, and have DSP set to stereo output with 192K max for any frequencies over 192K. The display on the Mytek does correctly show DSD for the DSD64 files and HDSD for the DSD128 files.
If the frequencies are not limited to 192K, I keep getting an error of 352K, etc won't play on the Mytek. This is the same error I get when using Firewire, so somewhere a frequency greater than 192K is being sent to the Mytek and causing a freeze.
The only problem I'm not seeing is the occasionally buffer limit resulting in a dropout of 3 sec or so when playing the DSD128 files, despite setting the output mode buffering to 5 sec. (It is worse if left at 1 sec.) This may be due to running the std USB2 output on the PC motherboard.