Welcome to the course!

Extreme Gradient Boosting with XGBoost

Sergey Fogelson

Head of Data Science, TelevisaUnivision

Before we get to XGBoost...

• Need to understand the basics of
  • Supervised classification
  • Decision trees
  • Boosting

Supervised learning

• Relies on labeled data
• Have some understanding of past behavior

Supervised learning example

• Does a specific image contain a person's face?
• Training data: vectors of pixel values
• Labels: 1 or 0

Supervised learning: Classification

• Outcome can be binary or multi-class

Binary classification example

• Will a person purchase the insurance package given some quote?

Multi-class classification example

• Classifying the species of a given bird

AUC: Metric for binary classification models

• Area under the ROC curve (AUC)
• Larger area under the ROC curve = better model

Accuracy score and confusion matrix

• Confusion matrix

• Accuracy $$\frac{tp+tn}{tp+tn+fp+fn}$$

Review

• Supervised Learning with scikit-learn

Other supervised learning considerations

• Features can be either numeric or categorical
• Numeric features should be scaled (Z-scored)
• Categorical features should be encoded (one-hot)

Ranking

• Predicting an ordering on a set of choices

Recommendation

• Recommending an item to a user
• Based on consumption history and profile
• Example: Netflix

Let's practice!

Extreme Gradient Boosting with XGBoost