It absolutely affects the sound the woofer produces, if you like the way the speaker sounded before you undertook the repair job you will need to put the clay back in when you are finished to restore the sound you are used to. It depended on the manufacturer, and if they were trying to use one woofer in as many models as possible. The resilient mass located near the voice coil would tend to induce a mechanical rolloff of the woofers midrange output and maybe part of the crossover design in this speaker.