I'm with Tyson on this one. Maybe look at it another way: considering this is a hobby where folks go through gear at varying intervals, of all the stuff you currently have, if you had to pick what you would replace, first to last, trying to upgrade what would that list look like? As you mention you absolutely love your speakers, and are at least satisfied with your other gear, it reads to me like of all the things you would change you would probably get rid of your speakers last (which from a "if I had it to do all over again" approach to this hobby is actually a really good thing!).
So if your speakers are going to be the base upon which you continue to upgrade your system, going all-in on your speakers is going to be an investment that is going to last for the longest time. It also means that you will also take care of that "what if" that is gnawing at you for not a lot of additional investment, which is for sure worth something. As Tyson also pointed out, upgrading your speakers as far as they'll go (within practical reason before you would want to move on to a different speaker model) will just make all the other upgrades you might do in the future on your upstream gear even better.
If we are just musing in general about how you should spend your next audio hobby dollars, I don't know the exact cost of the upgrades to all three of your speakers, but I might offer an alternative that in my opinion might be a real coin toss to upgrading your speakers now or later. If you already have (and have built, meaning you know how and can do so) one of Danny's subs, and your upgrades would be in the ballpark of $500 (the ballpark cost of another sealed servo sub driver and amp), I might go for a second sub first, then upgrade your speakers. Going dual subs, run in stereo, in my opinion, would give you a lot of bang for your audio buck. Also, even if you move on to other speakers in the far-away future, I can only think of a few circumstances in which your sub(s) won't stay with you. For what they do, Danny's subs have few peers, and arguably none better, regardless of price.
Congratulations on being on the short(er) path to dialing a system you love rather than having to go at it from scratch.