Dave, you are oh so correct.
I often liken
Serious Listening to
High Performance Driving (like F1) where inches and timing can make substantial differences in results.
In audio I call it
High Performance Listening.
It is the precision alignment, positioning and placement of YOU, your Speakers, and the relative positions of both of those to the surrounding boundaries.
The more precisely we can adjust the distances and relationships, the more accurate the reproduction of the sonic information. The general goal being purity to the original even, and minimal affect from everything else including the listening room
While I certainly wouldn't want to discourage anyone from always trying to experiment with improving those relationships, I must say that the journey can often take some time, but the results are oh so worth it.

That is why it is interesting to hear peoples comments at shows where the room being used is MUCH different than most anyone's real listening room.
To simply get things sounding decent (in but a small area) is a remarkable accomplishment. Kudos to the experienced "set up" Maestro, who can get it close the first time, and get it "right" before the end of the show.
I often hear Customers and Friends mention how their speakers and systems are still sounding BETTER even after years of ownership. Those small but subtle changes and improvements are often the results of the
very small changes you mention.
And your advice on that CD or "a" specific cut that is used as a
reference for system set up and adjustment is always a good idea.