Multiple Comparisons (3 of 3)
Next section: Orthogonal comparisons
There is room for argument, however, about whether it should be strictly
controlled, or whether it should be allowed to rise slightly.
One consideration is the definition of a family of comparisons. Let's say you
conducted a study in which you were interested in whether there was a difference
between male and female babies in the age at which they started crawling. After
you finished analyzing the data, a colleague of yours had a totally different
research question: Do babies who are born in the winter differ from those born
in the summer in the age they start crawling? Should the EER be controlled
or should it be allowed to be greater than 0.05? A compelling argument can be
made that there is no reason you should be penalized (by lower power) just because
your colleague used the same data to address a different research question.
Next section: Orthogonal comparisons