Post-hoc testing

Inference for Numerical Data in R

Mine Cetinkaya-Rundel

Associate Professor of the Practice, Duke University

Which means differ?

Two sample t-tests for differences in each possible pair of groups
Multiple tests $\rightarrow$ inflated Type 1 error rate
Solution: use modified significance level

Multiple comparisons

Testing many pairs of groups is called multiple comparisons
The Bonferroni correction suggests that a more stringent significance level is more appropriate for these tests
- Adjust $\alpha$ by the number of comparisons being considered
- $\alpha^\star = \frac{\alpha}{K}$, where $K = \frac{k (k-1)}{2}$

Pairwise comparisons

Constant variance $\rightarrow$ re-think standard error and degrees of freedom: Use consistent standard error and degrees of freedom for all tests
Compare the p-values from each test to the modified significance level

Let's practice!

Inference for Numerical Data in R

Preparing Video For Download...