I was planning on not having a high pass filter for my midrange driver in a 3-way design while having a low pass filter on the woofer. Based on what you just said, this would be a bad idea because the woofer would be out of phase from the midrange at specific frequencies. Correct?
If the drivers had flat FR a cople octaves above and bleow the crossover point then yes, the midrange driver would stay flat while the low pass on the woofer would rolloff the woofer, so you would have unequal change in attenuation, which equals unequal rotation of phase. But you could have phase coherence if the midrange driver had a natural rolloff (without electrcial filter) to match the rolloff of the woofer with electrical low pass filter. Acoustic response is what really matters. Whatever causes attenuation also causes corresponding phase rotation. In this case, the most likely method by which the midrange would roll off with decreasing frequency is loss of acoustic impedance, meaning the cone is too small to grip the air at the low velocity of low frequency. This is exactly why larger cones are used to play bass freqs and smaller for treble. A small midrange cone, like 2" will roll off significantly at a frequency that could be used for a crossover. But it s rarely perfect and some kind of electrical filter is usually added as high pass or as EQ to tweak the response a little to better match the other drivers response. But there are some such designs like that. See Troels Gravesen DTQWT, no high pass filter on the 8" midrange.
Also when I’ve been saying “at the crossover point” I should have been saying “at the roll off point”.
The rolloff point, also called the "knee" of a high pass or low pass filter is -3dB. This is the nominal frequency of a filter outside the loudspeaker world, and early crossover designers used this standard to cross drivers at -3dB, with such XO designs being called Bessel, Butterworth, etc. but there are some problems with crossing at -3dB, which were solved by
Siegfried Linkwitz and Russ Riley. Most modern loudspeaker crossovers take advantage of their invention and cross at the -6dB point with Linkwitz-Riley crossover, aka "LR4" abbreviation to mean Linkwitz Riley 4th order XO.
In your case, before you decide midrange gets no high pass, measure all the drivers electrically and acoustically to determine their natural behavior without electrical filter. Then import the data into a loudspeaker design simulator (like X-Sim) to assist in choosing the electrical filters and physical arrangement of drivers that will give you the acoustic response you want. Then, build it, listen, cry, measure to find out what went wrong, repeat.