The rca outputs are probably still going through an output buffer (it might be directly off the chip? Jason will have to chime in on that). A headphone output is really just a buffer with more current capacity that comes after the pot. If you're driving into a high impedance load (such as the input of an amp or powered speaker), then the current capacity isn't used and the differences from the rca output would be marginal. We're not talking about equipment in the kilobuck range here; do you really think it's going to make a difference?
Mini-xlr connectors would be a terrible design choice because almost nobody uses those, and then you would need a specialized cable. Furthermore the uDSD doesn't have balanced outputs, so even if they slapped the mini-xlr jacks onto the case, you'd be bootstrapping with single-ended output anyways.
