Why the Published Literature Contains Overestimates of Effect Sizes

---

A research report containing a significant effect is much more likely to be published than is a research report containing a non-significant effect. This publication "bias" means that the published studies are a very select set of the studies actually conducted. Assume two investigators (A and B) are researching topics for which the difference between population means is 0.75 standard deviations; both experimenters use the same sample size of 15. The power for this experiment (using the 0.05 significance level) is 0.50. Assume that Investigator A's experiment produced a significant result but Investigators B's experiment did not. The effect size in Investigater A's sample was somewhat higher than the population effect size, and this result is publishable. Invesitgater B's effect size was somewhat lower than the population effect size, and this result is not publishable. In general, studies which, by chance, find effect sizes larger than the population effect size are much more likely to be published than are studies that, by chance, find effect sizes smaller than the population effect size. For more details see Lane & Dunlap, 1978.