A dentist would like tell me to fill it but I'd like your advice...

I have an 11' w x 13' deep room with 9' ceilings. I have a bay window behind me and the speakers are along the opposite wall. There is a wide doorway on the left side wall as I'm sitting on the couch.
The wall where the speakers are has a large closet-sized cavity behind it where there are some stairs that go to 2nd floor of my house. The only thing dividing my sound room from this cavity is standard sheetrock and the 2x4's it's mounted to. There is no insulation at all. I think this may be causing some bass reinforcement in my room but not in a good way.... kind of a slow, pudgy way.
I had the whole room setup 180* opposite the way it is now and I *think* the bass was more detailed. It's hard to A/B this situation.

I can actually cut a hole and get into the space and fill it with insulation or water balloons... or just about anything. I'm not going to turn it into a finished space, though.... too much work, too little time.
Any thoughts on this aspect of the room? What does a huge cavity under stairs do typically? Can I somehow use it to improve my sound?
I'd like to remove the sheetrock, cover it with fabric and fille the cavity with fiberglass as an experiment... but that's a whole lotta work for a complete unknown.
Thanks for suggestions.
-C