I had actually just planned to use hole saws (I'm guessing that will make any woodworkers here cringe). Would a circle jig be preferable? I'm not familiar, but then I'm not very familiar with any of this. Definitely want to have the smoothest cuts I can get, and am open to getting the circle jig you referenced.
I haven't found hole saws to be accurate enough. A Jasper jig is adjustable in 1/16" increments. I've found this to be close enough for speaker holes. If the diameter in the plans is between to 1/16" marking, use the larger position. The markings on the jig are designed for use with a 1/4" bit. You can use any bit you want but you have to compensate for the diameter of the bit. It's a lot easier to use a 1/4" bit. I prefer an upcut spiral bit with a plunge router base. Secure the baffle to a sacrificial board. Double stick tape works well for this. Make sure both the baffle and plug will be secure to the backer when you cut all the way through.
The way I use the jig for the tweeter holes is:
1 - Mark the center of the driver hole and with a 1/8" bit, drill a pilot hole.
2 - set the router with the bit installed on the baffle and push down until the bit is touching the baffle. Lock the plunge base so the depth doesn't change.
3 - put the tweeter flange on the plunge base depth stop and lower the stop rod until it is tight on the tweeter flange.
4 - lock the depth stop rod so it won't move. You have now set the depth for the flange rabbet.
5 - Put the guide pin in the Jasper jig for the size of the outside diameter of the tweeter hole.
6 - place the jig with the guide pin in the pilot hole.
7 - turn the router on and plunge it to the depth the stop rod is set at then cut a rabbet.
8 - Set the guide pin in the jig for a 1/4" smaller diameter circle and cut another rabbet.
9 - Continue cutting smaller diameter rabbets until the ledge is wide enough for the tweeter flange.
10 - set the guide pin in the jig for the diameter of the through hole then cut it.
11 - cut or sand the offset for the connectors to fit through the hole. I use a Dremel with a sanding bit.