You can go with a thicker front baffle if you want. The gains with the N3 will not be as great as with speakers that have woofers that are counter sunk requiring a lot of baffle thickness be milled away. Since the N3 woofers surface mount you don't loose baffle thickness. Also, metal frame woofers tend to transmit a resonance to the front baffle easily, so thicker baffles really help reduce that. The N3 woofers use a non-resonant polymer frame so it is not as much of an issue.
Just be sure that as the baffle thickness increases that you add a larger and larger radius on the back side of the woofer hole. The bigger the better.