Overview
This turns a test result into a clear decision, and also checks whether the effect is big enough to matter in practice, not just statistically significant.
Enter the p-value, your alpha, the effect size, the sample size and the smallest effect you'd consider meaningful.
Worked examples
| Inputs | Result |
|---|---|
| p-value (0-1)0.032Alpha (%)5Effect Size0.42Sample Size (n)180Practical Threshold0.2 | Decision: Reject H0p-value: 0.03 vs alpha 5%Effect size: 0.42Practical threshold: 0.2Practical significance: Yesn=180 |
Show 2 more examples
| p-value (0-1)0.2Alpha (%)5Effect Size0.1Sample Size (n)50Practical Threshold0.2 | Decision: Fail to reject H0p-value: 0.2 vs alpha 5%Effect size: 0.1Practical threshold: 0.2Practical significance: Non=50 |
| p-value (0-1)0.001Alpha (%)5Effect Size0.05Sample Size (n)10,000Practical Threshold0.2 | Decision: Reject H0p-value: 0 vs alpha 5%Effect size: 0.05Practical threshold: 0.2Practical significance: Non=10,000 |
How it works
The result is statistically significant if p < alpha. It's practically significant if the effect size is at least your practical threshold. With large samples, tiny effects can be statistically significant but unimportant, which is why both checks matter.
Practical tips
- p = 0.032 at a 5% alpha with an effect of 0.42 against a threshold of 0.2 is significant both ways.
- Set the practical threshold before looking at the data.
- A non-significant result doesn't prove there's no effect. The study may have been too small.
Frequently asked questions
What's the difference between statistical and practical significance?
Statistical significance says the effect probably isn't chance. Practical significance says it's large enough to matter.
What effect size should I use?
Whatever fits your test, such as Cohen's d or a percentage difference, with a matching threshold.
Why is sample size shown?
Large samples make even tiny effects statistically significant.