Hi Oracle,
There has been a bit on this topic for some time. Do a search on 'switch-on' and you will find quite a bit of reading. This has been discussed and researched by me and by Ginger, and there's not much can be done without destroying the sonics of the amplifier.
I believe your speaker is too small for your amplifier. Maybe a 25W AKSA would be a better match. My understanding is that the Fostex you are using is rated to about 25W rms continuous, and the spike at switch-off it 3/4 second at 12V, which is around 24W, though not over the full 3/4s. Your voice coil will be fine, and in truth there will be no damage to the suspension either as it's slow speed - not a 40Hz back and forth pulsation.
Hope this is helpful,
Cheers,
Hugh