I used a 2 flute endmill on the back panel for the RCA holes.  The larger holes (for the tubes) I used a step drill.  Make sure your mount the panel rigidly before drilling.   Try to use a drill press or milling machine.  I had my top panel in a machine vise with some 2"x4" wood pieces underneath.  The step drill will go through the under laying wood without a problem.  Makes for a perfect circle.  You may need to do some deburring,  I would get a deburring tool for this as it will make for a real nice edge compared to sandpaper.