It seems everyone is assuming the sonic compromises with USB are derived from the electronics. Please consider the cable.
When I got my BDA-3, I tried 3 different USB cables between the BDP-2 (with IAD card) and DAC. For PCM Hi-Rez, I also compared the USB cables to my AES cable, a Hi-Diamond Big cable. Without any doubt, there were significant differences between the different USB cables. Two USB cables, the Wireworld Platinum Silver and WyWires Silver, had wonderful space and air, with the prize going to Wireworld Platinum for top to bottom punch and smoothness at the same time. A Shunyata Venom USB had less detail, much less "air" and the music was less engaging. I'm now using it on my computer-based system with an iFi DAC.
It's so easy to compare cables with a service like The Cable Company. I would urge anyone trying to get the best from these components to try before you buy. I might have ended up with a different selection if I had continued the search. But on my Bryston gear, I settled on the Wireworld Platinum, and of course kept my Hi-Diamond AES. I still prefer PCM/Hi-Rez thru the AES (by a hair), but without a remote control to change inputs for my BDA-3, when I'm cycling through my full library including DSD and PCM files, and feeling too lazy to get up and switch inputs, I just keep it on the USB input with negligible compromise.