Other than driving speaker harder than the drivers can take are there any other variables that cause a speaker to distort?
There are many sources of potential distortion in an audio system. From a speaker perspective, there are four main potential issues.
Every driver has an XMAX - the maximum cone excursion possible before the driver no longer performs within specs or actually bottoms out. But this would only occur when you are "driving the speakers harder than the drivers can take."
Another possibility is debris lodged between the voice coil and the magnet. This would produce a "scratching" sound as the driver moves in and out.
The other potential issues relate to a failure of the driver itself. The spider that attaches the rear of the speaker cone to the frame is glued in place. If the adhesive fails, the driver will tend to produce a "flapping" sound. Other damage can be a surround that has failed or come loose. You could normally see that with a visual inspection.
Finally, the gasket material between the driver and the cabinet can fail to make a seal. This results in an air leak and produces a sound similar to someone passing gas when the system is driven hard. This is usually associated with large woofers. Pressing on the woofer frame to force the driver into the gasket material, or covering the mounting holes with your fingers can stop the sound of an air leak temporarily and is an indication the gasket material needs to be replaced and the driver re-mounted.
The latter three of these would produce significant issues that one would perceive as more severe than simple distortion.
But providing these three situations did not exist and you are not driving the speaker drivers beyond their XMAX, the source of any distortion is almost always elsewhere in the system. A speaker normally works or it does not and it merely reproduces what it is fed. Distortion can be produced at all points from the source to the speaker. And once it is introduced, it can't be removed.
We have investigated many distortion issues over the years. The source is almost never with the speakers themselves and, if it is, it is usually a case of driver failure that results in sound that is so degraded, you would not mistake it for simple distortion. You would definitely know the speaker is damaged.