At What Level is H0 Really Rejected? (2 of 3)

---

He or she would have no more basis to doubt the validity of the null hypothesis than if p had been 0.482. The conclusion would be that the null hypothesis could not be rejected at the 0.05 level. In short, this approach is to specify the significance level in advance and use p only to determine whether or not the null hypothesis can be rejected at the stated significance level.

Many statisticians and researchers find this approach to hypothesis testing not only too rigid, but basically illogical. Who in their right mind would not have more confidence that the null hypothesis is false with a p of 0.0001 then with a p of 0.042? The less likely the obtained results (or more extreme results) under the null hypothesis, the more confident one should be that the null hypothesis is false. The null hypothesis should not be rejected once and for all. The possibility that it was falsely rejected is always present, and, all else being equal, the lower the p value, the lower this possibility.