Specific Comparisons among Means: Overview (2 of 3)

---

A very different and much less powerful procedure must be used if the comparison is unplanned. It may seem that it should not matter whether or not a comparison is planned in advance. After all, the the same data are analyzed regardless of whether or not the comparison is planned. It does matter, however, because if an experimenter looks at the data and then chooses a comparison, he or she will almost certainly choose to compare the means that differ the most. This is tantamount to doing all possible comparisons among means, a procedure that, by capitalizing on chance, produces an inflated Type I error rate.

The Scheffé test is a procedure that allows one to test unplanned comparisons among means without inflating the Type I error rate. Scheffé's test gives the freedom to test any and all comparisons that looks interesting. However, this great flexibly has a cost: Scheffé's test normally has very low power.