Probably true but you can do a lot of nice stuff to a small house without spending a lot.
My entire house has 7 windows. I went with the Marvin Integrity glider with LoE 366 w/Argon - total cost $3352.27
Replacing the windows was a no-brainer. Did I mention new windows are quiet? You wouldn't believe how low the ambient sound level is with new windows (think music room).
Most efficient windows: fixed (not every window in the house needs to open, most of ours don't).
Next most efficient windows: casement/awning (hinged like a door, the seals work better).
Worst efficiency windows: sliders/double hung (no compression of seals along the sliding portions).
But yes, good windows are quiet. We bought poor man Pella (the ones with 13/16" separation between glazing which are great) windows (Norco, now called Jel-Win) and can barely hear 40 mph wind outside.