The Precise Meaning of the Probability Value (1 of 3)
There is often confusion about the precise meaning of the probability computed in a significance test. As stated in Step 4 of the steps in hypothesis testing, the null hypothesis (H0) is assumed to be true. The difference between the statistic computed in the sample and the parameter specified by H0 is computed and the probability of obtaining a difference this large or large is calculated. This probability value is the probability of obtaining data as extreme or more extreme than the current data (assuming H0 is true). It is not the probability of the null hypothesis itself. Thus, if the probability value is 0.005, this does not mean that the probability that the null hypothesis is true is .005. It means that the probability of obtaining data as different or more different from the null hypothesis as those obtained in the experiment is 0.005.
The inferential step to conclude that the null hypothesis is false goes as follows: The data (or data more extreme) are very unlikely given that the null hypothesis is true. This means that: (1) a very unlikely event occurred or (2) the null hypothesis is false. The inference usually made is that the null hypothesis is false.