Indeed, it is impossible to control for the variability of systems. At the very least, each of the participants will discover whether or not cryo treatment is worth the expense for him/her. If the results are such that 75% correctly identify which cables are the same and prefer the pair/pairs that were treated, then I think the uncontrolled variable is a moot point. If the results are statistically meaningless (or worse ), then I agree that asserting the treatment to be ineffective isn't necessarily supported by the data.
I agree with Jim about not communicating during the data gathering phase of these experiments. We cannot enter the listening evaluations with any such information.
Everyones' system is different, thus producing an uncontrolled, but constant variable. If the changes wrought by cryo are audible on all systems then this is irrelevant.
Of course there are other uncontrolled variables that people are skirting.
1. the individuals' acuity
2. the individuals' attitude
3. the individuals' intellectualy honesty
These are a few of them.
We decided early on that there would be NO communication of the results until a statistically meaningful result was determined.
No test is perfect. True ABX testing is one of the most flawed tests available because it does not allow for unstressed observation plus other issues.
We're trying to have some fun here. If 8 out of 10 get the identifications correct we know one result. If 3 out of 10 get them correct we have another result and so on. Let's try not to overanalyze an admittedly non-scientific test. A true scientific test would use a random group of participants not knowing what their focus group would be testing in advance to help eliminate prejudices.
Dave