Beg or borrow more efficient speakers to make sure the amp is working properly. Seems like you're into the synergy (proper/improper equipment matching) game big time. Analogy: You could have the best 1.8L engine in the world, but it won't show itself off trying to pull a fully loaded 18 wheel semi-truck/trailer. I thought the Vandy's might work OK at low volumes, but evidently they're sucking up too much power.
Remember that wattage has a logarithmic relationship to dBs (how loud we hear). Every 3 dB increase seems half again louder but takes twice the power and every 10 dB seems twice as loud, but takes 10 times the power. Realize that most live music peaks between 105 and 110 dB, but the peaks can vary greatly. Typically (using vast generalizations here) rock has 5 dB peaks, jazz 15 dB, and classical 30 dB. Here's a quick table to compare watts with gain:
0.01 w = -20 dB of gain
0.1 w = -10 dB
1 w = O dB (by definition from how speaker efficiencies are rated)
2 w = 3 dB
5 w = 7 dB
8 w = 8 dB
10 w = 10 dB
13 w = 11 dB
16 w = 12 dB
20 w = 13 dB
50 w = 17 dB
80 w = 18 dB
So you can see that in the extreme case (classical music) average realistic listening levels are about 75 dB (comfortable to still carry on a conversation), but need 1,000 times the power to handle the peaks. With your Vandy's it takes about 0.04 watts to reach 75 dB, but 40 watts to reach the realistic peaks (and some would argue that actual/instantaneous peaks are 130 dB that would require 10,000 watts for the Vandy's!). That's what the simply math says, however if you go back to the small engine/big load analogy above it's even worse in the real world where big vehicles simply tend to have poor acceleration due to their mass and the basic effects of momentum. And music is all about peaks and acceleration. Fortunately for small tube amp owners, tubes tend to overload very gracefully, so they can "get around" these peak power demands with use of higher efficiency speakers being the other half of the solution.
But in life there's always a catch (or two). Deep bass takes lots of energy, so high efficiency speakers (like Omega designs) tend to not even try to reach say 20 - 50 Hz (without adding a subwoofer). I tried to join the SET scene for years (being fascinated with how the small/simple design could work), but finally gave up because: 1.) I believe bass is essential, even foundational and wanted the full frequency spectrum to come from one point; 2.) Nearly all high efficiency speaker designs were colored (midrange honk, cabinet noise from insufficiently braced large cabinets, etc.); and 3.) The designs that tried to do bass were uniformly huge (plus expensive) and tried to offset bloated tube bass with the still weak speaker bass capabilities (complementing each others' distortions).
That's where Louis' approach makes so much sense: superior drivers (where can a 'poor' driver hide in a single driver design?); attention to cabinet building; and leaving deep bass to the subwoofers.
