The Tukey HSD is not a Post Hoc Test
David M. Lane
Rice University

Suppose a researcher were interested in all pairwise differences among four conditions and, before collecting the data, decided to use the Tukey hsd to test these differences for significance. There is no sense in which this use of the Tukey hsd could be considered post hoc (defined as "formulated after the fact," Mirriam-Webster) rather than a priori (defined as "formed or conceived beforehand," Mirriam-Webster).

Although the most common approach in this situation is to do an ANOVA and conduct the Tukey hsd only if the ANOVA is significant, the Tukey hsd controls the family-wise error rate and is sound whether or not an ANOVA is performed (Lane, 2010, Wilkerson, 1999). Neither approach is uniformly more powerful since their relative power depends on the pattern of population means. It is possible for a Tukey comparison to be significant even if the ANOVA is not significant just as it is possible for the ANOVA to be significant without any of the Tukey comparisons being significant.

If, instead of planning in advance to use the Tukey hsd, suppose a researcher found a significant ANOVA and, after examining the means, decided to do the Tukey hsd rather than, say, the Bonferroni correction. This use of the Tukey hsd would, indeed, be post hoc but would not fully control the family-wise error rate.

The field of statistics is confusing enough without giving words different meanings in different contexts. To use the term "post hoc" to describe both the Tukey hsd and the Scheffé test does not lead to conceptual clarity.

References

Lane, D. M. (2010) Tukey's Honestly Significant Difference (HSD). In N. J. Elkind (Ed.) Encyclopedia of Research Methods, Sage Publications.

Wilkinson, L. and the Task Force on Statistical Inference, APA Board of Scientific Affairs (1999). Statistical Methods in Psychology Journals. Guidelines and Explanations. American Psychologist, 54, 594-604.