Its less effective than you'd probably expect. I've used thin pieces of tape to cover the tiny speakers that beep every time they scan a product. it actually makes them much quieter and reduces the frequencies that are able to pass through.
To be fair, the openings on these speakers are quite small, (1-2mm) and a larger suspended membrane, might work somewhat better, but would still be largely restrictive, not to mention more easily damaged by things like vacuums, children's fingers, sharp objects, etc. Esp as the membrane deteriorates over time You're much better off just having them play directly into open air. Anything else would make them sound muffled/veiled/distorted.