Regression review

Extreme Gradient Boosting with XGBoost

Sergey Fogelson

Head of Data Science, TelevisaUnivision

Regression basics

  • Outcome is real-valued

Extreme Gradient Boosting with XGBoost

Common regression metrics

  • Root mean squared error (RMSE)
  • Mean absolute error (MAE)
Extreme Gradient Boosting with XGBoost

Computing RMSE

Actual Predicted
10 20
3 8
6 1
Extreme Gradient Boosting with XGBoost

Computing RMSE

Actual Predicted Error
10 20 -10
3 8 -5
6 1 5
Extreme Gradient Boosting with XGBoost

Computing RMSE

Actual Predicted Error Squared Error
10 20 -10 100
3 8 -5 25
6 1 5 25
  • Total Squared Error: 150
  • Mean Squared Error: 50
  • Root Mean Squared Error: 7.07
Extreme Gradient Boosting with XGBoost

Computing MAE

Actual Predicted Error
10 20 -10
3 8 -5
6 1 5
  • Total Absolute Error: 20
  • Mean Absolute Error: 6.67
Extreme Gradient Boosting with XGBoost

Common regression algorithms

  • Linear regression
  • Decision trees
Extreme Gradient Boosting with XGBoost

Algorithms for both regression and classification

1 https://www.ibm.com/support/knowledgecenter/en/SS3RA7_15.0.0/ com.ibm.spss.modeler.help/nodes_treebuilding.htm
Extreme Gradient Boosting with XGBoost

Let's practice!

Extreme Gradient Boosting with XGBoost

Preparing Video For Download...