A
statistical hypothesis test is a method of making statistical decisions using experimental data. In
statistics, a result is called
statistically significant if it is unlikely to have occurred by
chance. The phrase "
test of significance" was coined by
Ronald Fisher "Critical tests of this kind may be called tests of significance, and when such tests are available we may discover whether a second sample is or is not significantly different from the first."
[1]Hypothesis testing is sometimes called confirmatory data analysis, in contrast to exploratory data analysis. In frequency probability, these decisions are almost always made using null-hypothesis tests; that is, ones that answer the question Assuming that the null hypothesis is true, what is the probability of observing a value for the test statistic that is at least as extreme as the value that was actually observed?[2] One use of hypothesis testing is deciding whether experimental results contain enough information to cast doubt on conventional wisdom.
Statistical hypothesis testing is a key technique of frequentist statistical inference, and is widely used, but also much criticized. The main alternative to statistical hypothesis testing is Bayesian inference.
The critical region of a hypothesis test is the set of all outcomes which, if they occur, will lead us to decide that there is a difference. That is, cause the null hypothesis to be rejected in favor of the alternative hypothesis. The critical region is usually denoted by C.
