Experimentwise Error Rate

When a series of significance tests is conducted, the experimentwise error rate (EER) is the probability that one or more of the significance tests results in a Type I error. If the comparisons are independent, then the experimentwise error rate is:

where

α_ew is experimentwise error rate

α_pc is the per-comparison error rate,

and c is the number of comparisons. For example, if 5 independent comparisons were each to be done at the .05 level, then the probability that at least one of them would result in a Type I error is:

1 - (1 - .05)⁵ = 0.226.

If the comparisons are not independent then the experimentwise error rate is less than

.

Finally, regardless of whether the comparisons are independent,

α_ew ≤ (c)(α_pc)

For this example, .226 < (5)(.05) = 0.25.