More non-parametric tests: Spearman correlation

Performing Experiments in Python

Luke Hayden

Instructor

Correlation

Basis

  • Relate one continuous or ordinal variable to another

  • Will variation in one predict variation in the other?

 

Pearson correlation

  • Based on a linear model
Performing Experiments in Python

Pearson vs Spearman correlation

Pearson correlation

  • Parametric

  • Based on raw values

  • Sensitive to outliers

Assumes:

  • Linear, monotonic relationship

Effect measure

  • Pearson's r

Spearman correlation

  • Non-parametric

  • Based on ranks

  • Robust to outliers

Assumes:

  • Monotonic relationship

Effect measure

  • Spearman's rho
Performing Experiments in Python

Pearson vs Spearman correlation

Pearson's r: 1, Spearman's rho = 1 Scatter plot of sample distribution

Performing Experiments in Python

Pearson vs Spearman correlation

Pearson's r: -1, Spearman's rho = -1 Scatter plot of sample distribution

Performing Experiments in Python

Pearson vs Spearman correlation

Pearson's r: 0.915, Spearman's rho = 1 Scatter plot of sample distribution

Performing Experiments in Python

Pearson vs Spearman correlation

Pearson's r: 0.0429, Spearman's rho = 0.0428 Scatter plot of sample distribution

Performing Experiments in Python

Spearman correlation example

Scatter plot of Olympic athlete height over time

Performing Experiments in Python

Implementing a Spearman correlation

from scipy import stats
pearcorr = stats.pearsonr(oly.Height, oly.Weight)
print(pearcorr)

(0.6125605419882442, 7.0956520885987905e-190)

spearcorr = stats.spearmanr(oly.Height, oly.Weight)
print(spearcorr)

SpearmanrResult(correlation=0.728877815423366, pvalue=1.4307959767478955e-304)
Performing Experiments in Python

Let's practice!

Performing Experiments in Python

Preparing Video For Download...