Aside from such issues as native DAC sampling rates, etc., realize that upsampling is a means of enabling the use of better reconstruction filters. The Redbook sampling rate of 44.1 kHz means that your Nyquist frequency is 22.05 kHz, only 2 kHz above the commonly referenced 20 kHz upper limit to human hearing (though that's quite optimistic for most adults). That means that your reconstruction filter needs to be very steep indeed (high order) if you want to avoid rolling off any part of the audio band, which is essentially impossible to do without introducing ripple/ringing into the audio band, phase shift, or one of several other artifacts. By upsampling to a higher sampling rate, designers have far more room to maneuver in designing reconstruction filters that are steeper (to better remove HF artifacts), yet still introduce fewer deleterious side effects. (That barely scratches the surface of sampling theory and implementation, by the way, but Google is your friend...)
So as a takeaway - don't dismiss upsampling in it's entirety without understanding what reconstruction filter was engaged when you evaluated it. It may be that your NOS DAC threaded the needle in its reconstruction filter design and its compromises align with your preferences perfectly. But it may very well be that what you didn't like about a particular upsampling DAC was actually the reconstruction filter in use at the time, and thus not the enabling capability that is upsampling itself. Many DACs offer a small number of reconstruction filters with which you can experiment, but if you pursue computer audio, then you have the option of dozens of filters through applications like HQPlayer, etc.. But be warned - like speakers, or amps, or cables, or couple caps, or solder, it's yet another pile of never-ending choice to navigate. I've met computer audio aficionados that change their filter selection with the genre of music they are enjoying...
(Although automating this may not be far off, as I can imagine the day where Roon tells HQPlayer or it's own DSP engine what filter to use based on the genre to which a song belongs, or perhaps even the instrument mix.)