Sorry I'm late to this one but maybe this will help some future builders.
My recommendation for magnets is to install them from the rear side before glue up. Set up your drill press so that the holes leave minimal material between bit and face, I try to leave less than 1mm when I'm doing them. Once holes are drilled, dip each magnet in epoxy or even some thin ca and then tap them down to the face.. be sure to do this on a hard flat surface so not to punch the magnet through the face. If you are worried about it, you add some glue behind the magnet as well. Doiong them this way leaves the baffle surface untouched, no filling/sanding etc and no chance of bleed through down the road. Be sure to place them so that you leave clearence for frontside rouindovers if you plan to use them. Youi don't want the router bit coutting into the magnets

I've read that acoustically, you'll want a minimum of .5" rouinodver on front vertical edges although, I'm not sure how much difference you actually notice if you went smaller. Dont forget the rear roundover on the rear of the woofer cutout , again, I'd go with a .5"
If you had a circle jig for your router, I'd recommend cutting a plug to perfectly fill the tweeter through hole then recutting the tweeter recess. Once that was done, you could re-cut the through hole assuming you glued the plug into place. Agree with Peter, trying to sand the face of the tweeter recess is not going to be fun .
Youi may gain a little more room once the tweeter gasket compresses but it won't be a 1/16".
jay