Cautions in Interpreting Variance Explained (1 of 3)
Measures of the proportion of variance explained can be easily misinterpreted.
One mistake a researcher can make is to ignore the fact that the
levels of an independent variable chosen for inclusion in the study
can be a prime determinant of the proportion of variance explained.
For example, consider an investigation seeking to determine the proportion
of the variance in reading ability that can be accounted for by age.
If the experimenter chooses ages 5 years, 10 years, and 15 years for
inclusion in the study, then a large proportion of the variance will
be explained by age. On the other hand, if ages 10 years, 10.2 years,
and 10.4 years were included, only a very small proportion of the variance
would be explained by age. This is not a fault with the measure of effect
size per se; however, it represents one way in which the measure can
be misinterpreted.
A second problem with measures of the proportion of
variance explained is that there is no consensus on how big an effect
has to be in order to be considered meaningful. In some cases, effects
that when measured in terms of proportion of variance explained appear
to be trivial, can, in reality be very important.