The doubt and the qualms you have are understandable, especially when it comes to speakers. If it is any consolation, the newer GAPs, according to HiFi plus magazine in the UK, pull a major sonic trick, and this is with the price calculated in British pounds.
I have a pair of Gershman X-1s, which are now called X-2s with the respective subwoofers. I have thought about selling them, but I have not, mostly because I feel the Gershmans have their particular sonic signature.
When I think about the Odyssey amps you have, I believe that if you EVER get the GAPs, you will find the combination ravishing.
A particularly nice combination would be the GAPs with your amps and an ultra reveling or lighting fast preamp or a source such as a Wadia preamp+CDP.
PS: 5K is a lot of money, but the GAPs seldomly ever come to sale, besides, when it comes to everything, no investment, there is definetly no chance of gain.