Combining evidence from p-values

Foundations of Inference in Python

Paul Savala

Assistant Professor of Mathematics

p-values

A woman lifting a heavy weight.

Foundations of Inference in Python

Repeated experiments

  • Same experiment --> Different results?

People exercising with weights.

Foundations of Inference in Python

Different effect sizes

  • Different samples --> Different conclusions
  • Culprit: Effect size
Foundations of Inference in Python

Testing p-values

  • Fisher's method

    • Different samples/studies
    • Same null hypothesis
    • Different p-values
    • At least one should reject the null

  • Combines evidence from multiple studies

Foundations of Inference in Python

Fisher's method in SciPy

p_values = [0.052, 0.12, 0.09, 0.051]


fishers_stat, p_value = stats.combine_pvalues(p_values)


print(p_value < 0.05)

TRUE
  • Studies all testing the same null hypothesis
Foundations of Inference in Python

Fisher's method in SciPy

p_values = [0.01, 0.51, 0.81, 0.49]


fishers_stat, p_value = stats.combine_pvalues(p_values)


print(p_value < 0.05)

FALSE
Foundations of Inference in Python

Let's practice!

Foundations of Inference in Python

Preparing Video For Download...