Personally I would leave it on all the time especially if you say you can hear the difference when it has been on a while.
I use to have NuForce Ref 9SE V2 mono's, I left them on all the time as they produce very little heat.
In fact I never turn most things off in my Hi-Fi, they will always be ready when I want to use them and I listen daily. The Only things I turned off are the turntable as the motor would just keep turning and the tape deck because the capstans are constantly turning if the deck is on.
If you are still not sure about leaving them on all the time, contact NuPrime I find them to be very helpful well they was as NuForce anyway.