Before you glue the front baffle in place, you can inset them in the back of said baffle. Just carefully drill in a small amount to seat the magnet. If you use the good, strong magnets, they will hold through lots of wood. I don't know exact measurements to advise you to use, but others might.
Oops, what jay said....
ben