An interesting, and duly noted, open ended question, that many folks are taking in cool directions.
Tough one for me to answer, but the more I think about it, the more for me it is:
1 - speakers
2 - source
3 - amp
4 - room
ZIP up that firesuit Mark, here come the flames from the astute ROOM folks out there.

Allow me to express why, for me, this is how I stack rank.
I have made a wholesale shift in my system that was static for 10 years, over the last 3 years. Not a single cable remains from what I used to run. Period. Thanks AC!

Of the system I run at the moment, that for the most part has me very happy, the piece that started it all off is the speakers. The addition of the b200 Visaton driver on OB changed the way I perceive music. Call it the coherency of the single driver thang, call it the idiocy of some loony Canuck, whatever, it is what I then based my system on.
That led to my public amp swapping, that has astutely been called "using the downfall of the amp to match the downfall of the speakers" by a member I completely respect - I mean that, I take that as an accurate description of what I happen to call "synergy" of those open baffle speakers and the single ended Class A EL84 amp I use now.
Early on in the amp swapping arrived the SB3. Subsequent modding by Bolder and that signalled the sale of my tubed preamp, a DAC, a PS for the DAC, and various cables.
After the speakers, but before the source and amps, came my moving houses, and thus having a new, larger room to play in.
So, when I look at the order of how I settled in on the system I use today, at this point, I stack accordingly.
Would room treatments vastly improve the sound? Perhaps, but as it is currently an unfinished basement, with no walls etc, I am less likely to invest. Crap, I have to come up with 20K to finish the darn room, and more for a pool table, THEN I can sort out how to best treat the resultant room.
All the parts are important, but how you build your system likely defines their stack rank order, for you.
All the best,