Yup - the brown stuff is commonly used to glue the base of caps to a board. You see this a lot in CRT devices where very high B+ is present, perhaps as an added layer of insulation should something arc. Sometimes you'll also see a silicone caulk-like goo they smear on that hardens slowly over time.
When a cap dies it is usually from overvoltage or a dried out dielectric due to environmental stress (like putting a cap next to a heat sunk component), or simply age. Common visual symptoms are either a bulge on top of the cap along the "vent" ridges, or they'll leak along the rubber gasket and leads at the base of the cap. This leaked material is generally white to gray in color, and can be tougher to spot. If in doubt, remove the cap, and check it out of circuit.
-Richard