Statistical and Practical Significance (4 of 4)


The 95% confidence interval on the difference between means is:

-5.8 ≤ μcolor - μblack & white ≤ -0.9

and the 99% interval is:

-6.6 ≤ μcolor - μblack & white ≤ -0.1

Therefore, despite the finding of a "more significant" difference between means, the experimenter can be even more certain that the color displays are only slightly better than the black and white displays. The second experiment shows conclusively that the difference is less than 10 seconds.

This example was used to illustrate the following points: (1) an effect that is statistically significant is not necessarily large enough to be of practical significance and (2) the smaller of two effects can be "more significant" than the larger. Be careful how you interpret findings reported in the media. If you read that a particular diet lowered cholesterol significantly, this does not necessarily mean that the diet lowered cholesterol enough to be of any health value. It means that the effect on cholesterol in the population is greater than zero.