OK, I unplugged a HumDinger (I have 3) from a sub and tried it on the 4B3. To the best of my ability to tell, the HumDinger didn't reduce this particular noise. That makes sense to me, as the main element of the RM20 is a large toroidal transformer, which should isolate the 4B3 from any DC offset on the AC line.
(Need I say I think the HumDinger is a useful device when the problem is DC offset? HumDingers are needed accessories to my two JL Audio subs and a Marsh amplifier.)
I tried to measure the noise to say something more objective. However, it is too quiet for my RadioShack SPL meter to measure. Since I can't hear it from a meter away, probably I should just accept it and not be shipping a heavy amplifier around. Two things elevated this to even a mild concern: (1) the absolute silence of a previous Bryston amplifier, and (2) too much time on my hands, due to pandemic-related isolation.
Thanks, @RPM123, for the suggestions. I always appreciate people taking the time!
And James, if you have anything to add, please do chime in.