Having owned VS VR4 Gen III HSE loudspeakers, I can say adding Dacron fill to the port is quite simple and does not involve the sock whatsoever.
If the OP's VR4 Gen III speakers are the same as mine (and I believe the only difference is the Hovland caps in the HSE edition) all that's required is to remove the plastic rear speaker binding post plate by removing the screws that hold it in place.
Gently pull the plate away from the cabinet being aware of the wires that connect the binding posts to the crossovers. Nevertheless, you can pull it completely away from the cabinet.
From there, the port is easily accessed through the opening.
The sock remains untouched.