The Limits of Prediction

Introduction to Linear Modeling in Python

Jason Vestuto

Data Scientist

Interpolation

Scatter plot of monthly Dow Jones price index versus dates from 2013 to 2015

Introduction to Linear Modeling in Python

Interpolation

Scatter plot of daily Dow Jones price index versus dates from 2013 to 2015

Introduction to Linear Modeling in Python

Interpolation

Scatter plot of monthly Dow Jones price index versus dates from 2013 to 2015

Introduction to Linear Modeling in Python

Interpolation

Scatter plot of monthly Dow Jones price index versus dates from 2013 to 2015, with model line over-plotted in red, fitting the data reasonably well

Introduction to Linear Modeling in Python

Interpolation

Scatter plot of daily Dow Jones price index versus dates from 2013 to 2015, with model line over-plotted in red, fitting the data only in a few limited ranges of dates

Introduction to Linear Modeling in Python

Domain of Validity

  • zoom in: data looks linear
  • model assumption: a2*x**2 + a3*x**3 + ... = zero.
  • build a linear model: a0 + a1*x
  • zoom out: your model breaks
Introduction to Linear Modeling in Python

Extrapolating Too Far

Scatter plot of altitude versus distance, with black points following a linear trend and points extending to each side deviating strongly from linear shown in blue, all with model line over-plot and fitting only the central black points

Introduction to Linear Modeling in Python

Let's practice!

Introduction to Linear Modeling in Python

Preparing Video For Download...