Hi Antman,
If your sub does not have a bypass input that bypasses the sub's internal crossover, then I would set the sub's crossover frequency all the way up to make sure it is "out of the way" of the Denon's crossover.
As far as the input level of the sub, I probably would not set that all the way up (which might introduce more noise & distortion). I adjust mine to give me the approximate correct loudness with my receiver's sub level at 0dB, then use the receiver's sub level control to fine tune it.
You are correct, setting your L & R to Large would have nothing coming out of the sub in stereo mode, only when playing DVD, DVD-A or SACD that had its own LFE channel.
You said that the Denon showed nothing out of phase, have you physically traced the speaker connections to verify?