I don’t think you can explain the sound differences between tubes and solid state by focusing only on distortion. I’m sure that designers of both types of equipment would say that their goal is realism and accuracy…and yet even with the same goals, five different designers will likely yield five different sounds.
Is one designer right and the others wrong? Absolutely not, for the designers, it comes down to their individual understanding and therefore execution of realism (whether using tubes or transistors). For listeners, although we say we want realism, it usually comes down to preferences.
A friend of mine says that it can’t sound real unless the bass is pushing you out of your seat (decidedly unrealistic from my perspective). Another says that realism means almost painfully loud and the cymbals sound like they are inside your head. My own simplistic definition uses words like clarity and aliveness.
I find a fundamental aspect is to define the term distortion. For me, and typically, distortion is the deviance of the output from the input. You are quite right that typical harmonic and intermodulation distortions do not fully determine accuracy. There are other distortions, from RCA Radiotron Designers Handbook, 26 engineers, 1960, a list.
1. Non-linear distortion.......... harmonic distortion. Which harmonics are involved
2. Intermodulation distortion.......... sums and differences. Unrelated to harmonics of the instrument/voice.
3. Frequency distortion.......... This includes frequency response limitations/deviations, resonances etc.
4. Phase distortion.......... Basically a alteration between the fundamental and one of its harmonics. Especially if a higher order harmonic precedes the fundamental.
5. Transient distortion.......... Peaks in signal due to design weaknesses. Usually with designs incorporating global feedback
6. Dynamic range and its limitations.......... Basically minimal to maximum signal reproduction.
7. Scale distortion.......... Basically, does the reproduction in the room mimick the spl at the live event?
I have found the frequency response to be of major importance, both from a masking perspective and the opposite, thin, bright.
As mentioned, the output accuracy to input is the ultimate form of minimal distortion and naturalness. The problem is how does one check their designs for accuracy. If those 5 designer's produce 5 different "sounds", then one at most, and probably none actually performed any sophicated listening tests to establish accuracy. More than likely, none of the 5 designs are sonically accurate.
Then there is the problem of an individuals reference. We all hear differently, however, if we all had exactly the same musical reference, then we would all agree what was most accurate. However, we all have different live musical references. We also have many who don't wish accuracy/naturalness, but another standard. That is personal preference.
Then a recording may be off, and manipulation is necessary to bring some sort of naturalness back.
It certainly can be complicated. Personally, it took me some 5 1/2 years just to get my test speaker accurate. It took some 40 years to get the preamp, amplifier, and source truly accurate. The result is simply amazing.
After all that work, all I can say is, good luck to everyone.