Statistical and Practical Significance (3 of 4)
The experimenter presented this conclusion to management but
management did not accept it. The color displays were so dazzling
that despite the statistical analysis, they could not believe that
color did not improve performance by at least 10 seconds. The
experimenter decided to do the experiment again, this time using 100
subjects for each type of display. The results of the second
experiment were very similar to the first. Subjects were slightly
faster at making reservations on a color display (M = 504.7 seconds)
than on a black and white display (M = 508.1 seconds) . This time the
difference was significant at the 0.01 level rather than the 0.05 level
found in the first experiment. Despite the fact that the size of the
difference between means was no larger, the difference was "more
significant" due to the larger sample size used. If the
population difference is zero, then a
sample difference of 3.4 or larger with a
sample size of 100 is less likely than a sample difference of 3.5 or
larger with a sample size of 50.