You are not the first person that had to repair the drivers on these 25 year old speakers, plenty of people have removed the glued-on grills and replaced the drivers and grills.
But in order to communicate with hive mind of the universe it would be useful to know what speakers you have and a photo of the grill-cabinet interface.
Have you tried removing the felt? It may have been applied using rubber glue or a 3M industrial adhesive. The cabinet may have been finished with a high gloss nitrocellulose lacquer or a have a high gloss catalyzed clear coat.
Is the speaker bottom finished? That is a good place to try out glue removers like
3M™ Adhesive Remover Citrus Base to see if they dissolve the finish. And you may need a package of Q-Tips.
I would recommend taping the speaker's entire front baffle with Scotch Delicate Surface tape before starting any work.