Overview

When you run many tests, some will look significant by chance. The Bonferroni correction controls that by making each test stricter.

Enter the overall (familywise) alpha, the number of tests and the raw p-value of the test you're checking.

Worked examples

InputsResult
Familywise Alpha (%)5Number of Tests12Raw p-value (0-1)0.008Bonferroni alpha: 0.42%Adjusted p-value: 0.1Not significant after correction
Show 2 more examples
Familywise Alpha (%)5Number of Tests3Raw p-value (0-1)0.012Bonferroni alpha: 1.67%Adjusted p-value: 0.04Significant after correction
Familywise Alpha (%)1Number of Tests20Raw p-value (0-1)0.0001Bonferroni alpha: 0.05%Adjusted p-value: 0Significant after correction

Bonferroni alpha by number of tests

With the calculator's starting values, the bonferroni alpha is 0.42%. The table shows how it changes with the number of tests.

Number of TestsBonferroni alpha
60.83%
80.63%
100.5%
12starting value0.42%
140.36%
160.31%
180.28%

Other inputs: Familywise Alpha: 5% · Raw p-value: 0.008 0-1. Change them in the calculator above for your own numbers.

How it works

The corrected alpha is alpha ÷ the number of tests. Equivalently, the adjusted p-value is the raw p × the number of tests, capped at 1, which you compare with the original alpha. It's simple but conservative, so with many tests, methods such as Holm or false discovery rate control lose less power.

The formula

Adjusted p = min(1, p × tests)

Practical tips

  • With 12 tests at 5%, each needs p < 0.0042. A raw p of 0.008 adjusts to 0.096, which isn't significant.
  • With 3 tests, the threshold is 0.0167.
  • Decide the number of tests before looking at the results.

Frequently asked questions

When should I use Bonferroni?

When you make several comparisons and want to limit false positives across all of them.

Is it too strict?

It can be, with many tests. The Holm method is uniformly more powerful.

What is familywise error?

The chance of at least one false positive across all the tests.