Caps are important. But when the caps in the crossover begin to exceed the cost of the drivers you have to ask how far down the rabbit hole you really want to go?
Correct, and spending more doesn't mean your getting more.
Once you start to hit the $40-60 range, per cap, (below 12-15uF at least) there aren't many great gains in performance left to be made.
Often if you're spending more, it's because you're looking for a certain tone or feel that a capacitor provides, which will affect clarity & soundstage presentation.
Tin, aluminum, gold, silver and copper each have their own tonal characteristics, same goes for the use of wax, oil, paper, epoxy or other dielectric materials.
It's all about knowing what sort of sound you want out of the speaker and it's components, and what sorts of materials will get you there.