Multiple comparisons and corrections

Foundations of Inference in Python

Paul Savala

Assistant Professor of Mathematics

Getting the result we want

A group of business people standing around and talking.

Foundations of Inference in Python

Real-world example

A group of people sitting together at a table planning business strategy.

Multiple comparisons problem: Making comparisons until getting the result we want.

Foundations of Inference in Python

The role of alpha in inference

  • p-value: Probability of a result occurring at random
    • Unlikely for people to get better "at random"
  • $\alpha$: Strength of result needed for significance
  • p-value $< \alpha$: Conclude a relationship exists

A roulette wheel.

Foundations of Inference in Python

Correcting for multiple comparisons

  • Starting $\alpha = 0.05$
  • Number of comparisons = 50

  • Bonferonni-corrected $\alpha$ = $\displaystyle\frac{0.05}{50} = 0.001$

  • Conduct hypothesis tests with $\alpha = 0.001$

Foundations of Inference in Python

Let's practice!

Foundations of Inference in Python

Preparing Video For Download...