Cautions in Interpreting Variance Explained (3 of 3)

previous


In most cases, there is no absolute standard to go by, and there is nothing much one can do other than measure effect size relative to subject variability. However, it is important not to overlook instances in which effect size can be measured more directly. For example, Tullis (1981) found that improving the format in which the results of a computerized test of a phone line are displayed reduces the average response time by about two seconds. although this effect does not explain much variance, it is important since computerized testing is done literally millions of times a year. The cumulative time savings of using the better display is gigantic.

Finally, the sampling distribution of measures of effect size complex, and confidence intervals on these measures are rarely computed (although procedures are available, see Steiger, 2004). It is difficult to interpret an estimate of an effect size without knowing the range of plausible effect sizes. When small sample sizes are used, it is possible for the estimate of effect size to be very different from the actual effect size. Therefore, it is easy to make the mistake of thinking a small effect is actually a large effect.


previous