Assessing model performance

Intermediate Regression in R

Richie Cotton

Data Evangelist at DataCamp

Model performance metrics

  • Coefficient of determination (R-squared): how well the linear regression line fits the observed values.

    • Larger is better.
  • Residual standard error (RSE): the typical size of the residuals.

    • Smaller is better.
Intermediate Regression in R

Getting the coefficient of determination

library(dplyr)
library(broom)
mdl_mass_vs_length %>% 
  glance() %>% 
  pull(r.squared)
0.8226
mdl_mass_vs_species %>% 
  glance() %>% 
  pull(r.squared)
0.7163
mdl_mass_vs_both %>% 
  glance() %>% 
  pull(r.squared)
0.9694
Intermediate Regression in R

Adjusted coefficient of determination

  • More explanatory variables increases $R^2$.
  • Too many explanatory variables causes overfitting.
  • Adjusted coefficient of determination penalizes more explanatory variables.
  • $\bar{R ^ 2} = 1 - (1 - R ^ 2) \frac{n_{obs} - 1}{n_{obs} - n_{var} - 1}$
  • Penalty is noticeable when $R^2$ is small, or $n_{var}$ is large fraction of $n_{obs}$.
  • In glance(), it's the adj.r.squared element.
Intermediate Regression in R

Getting the adjusted coefficient of determination

library(dplyr)
library(broom)
mdl_mass_vs_length %>% 
  glance() %>% 
  select(r.squared, adj.r.squared)
  r.squared adj.r.squared
      <dbl>         <dbl>
1    0.8226        0.8212
mdl_mass_vs_species %>% 
  glance() %>% 
  select(r.squared, adj.r.squared)
  r.squared adj.r.squared
      <dbl>         <dbl>
1    0.7163        0.7072
mdl_mass_vs_both %>% 
  glance() %>% 
  select(r.squared, adj.r.squared)
  r.squared adj.r.squared
      <dbl>         <dbl>
1    0.9694        0.9682
Intermediate Regression in R

Getting the residual standard error

library(dplyr)
library(broom)
mdl_mass_vs_length %>% 
  glance() %>% 
  pull(sigma)
152.1
mdl_mass_vs_species %>% 
  glance() %>% 
  pull(sigma)
313.6
mdl_mass_vs_both %>% 
  glance() %>% 
  pull(sigma)
103.4
Intermediate Regression in R

Let's practice!

Intermediate Regression in R

Preparing Video For Download...