Macrojack.... I am completely with you on your dilemna. The encouragement I can give is buy good and highly sought after drivers, then if for some reason you had to scrap everything, you can hock the drivers at reasonable resale. Horns can always fetch good used prices as they almost never come up for sale since so few are made commercially and it is a real effort to negotiate custom made ones.
It boils down to your commitment to slugging it out, even when hurdles are hit. If you are willing to sell certain components that didn't work out and buy components needed to modify your initial idea, in a way that has hope of working, then it isn't so bad. I think we, the so inclined to diy/learn/build, are especially impowered today with the availability of modern digital crossovers (Behr DCX, DEQX, other pro versions) that makes try and error of crossover development much quicker and less painful.
Even if you don't ultimately use the digital crossover in the end result, the ability to experiment in near real time with time delays, slopes and points with a digital crossover and a set of amps (how about a cheaper receiver with multichannel analog inputs) to determine what is needed is greatly valuable. Then you can build the passive version to match the digital as best as you can.
It narrow the possibilities greatly. To me it is a game changing event. This justified the DEQX cost for me, since regardless of its marketing points it is an incredibly powerful development tool that puts the power of speaker development in my own hands. There is an associated learning curve, but it is much less than before in many ways, or at the very least enables the learning of the speaker design process which was much more difficult before.