What is a decision tree?

Extreme Gradient Boosting with XGBoost

Sergey Fogelson

Head of Data Science, TelevisaUnivision

Visualizing a decision tree

1 https://www.ibm.com/support/knowledgecenter/en/SS3RA7_15.0.0/ com.ibm.spss.modeler.help/nodes_treebuilding.htm
Extreme Gradient Boosting with XGBoost

Decision trees as base learners

  • Base learner - Individual learning algorithm in an ensemble algorithm
  • Composed of a series of binary questions
  • Predictions happen at the "leaves" of the tree
Extreme Gradient Boosting with XGBoost

Decision trees and CART

  • Constructed iteratively (one decision at a time)
    • Until a stopping criterion is met
Extreme Gradient Boosting with XGBoost

Individual decision trees tend to overfit

1 http://scott.fortmann-roe.com/docs/BiasVariance.html
Extreme Gradient Boosting with XGBoost

Individual decision trees tend to overfit

1 http://scott.fortmann-roe.com/docs/BiasVariance.html
Extreme Gradient Boosting with XGBoost

CART: Classification and Regression Trees

  • Each leaf always contains a real-valued score
  • Can later be converted into categories
Extreme Gradient Boosting with XGBoost

Let's work with some decision trees!

Extreme Gradient Boosting with XGBoost

Preparing Video For Download...