Introduction to boosting

Machine Learning with Tree-Based Models in R

Sandro Raabe

Data Scientist

single classifier

Machine Learning with Tree-Based Models in R

bagging or random forest

Machine Learning with Tree-Based Models in R

boosting 1

Machine Learning with Tree-Based Models in R

boosting 2

Machine Learning with Tree-Based Models in R

boosting 3

Machine Learning with Tree-Based Models in R

boosting 4

Machine Learning with Tree-Based Models in R

Adaboost

  • First famous boosting algorithm: Adaboost = Adaptive Boosting
  • Idea: Change weight of wrongly classified training examples in subsequent trainings

adaboost visualization 1

Machine Learning with Tree-Based Models in R

Adaboost

  • First famous boosting algorithm: Adaboost = Adaptive Boosting
  • Idea: Change weight of wrongly classified training examples in subsequent trainings

adaboost visualization 2

Machine Learning with Tree-Based Models in R

Adaboost

  • First famous boosting algorithm: Adaboost = Adaptive Boosting
  • Idea: Change weight of wrongly classified training examples in subsequent trainings

adaboost visualization 3

Machine Learning with Tree-Based Models in R

Adaboost

  • First famous boosting algorithm: Adaboost = Adaptive Boosting
  • Idea: Change weight of wrongly classified training examples in subsequent trainings

adaboost visualization 4

  • Improved by adding gradient descent
Machine Learning with Tree-Based Models in R

Coding: Specify a boosted ensemble

# Specify the model class
boost_tree() %>%

    # Set the mode
    set_mode("classification") %>%

    # Set the engine
    set_engine("xgboost")

Boosted Tree Model Specification (classification)

Computational engine: xgboost
  • Easy interface to boosting through tidymodels!
Machine Learning with Tree-Based Models in R

Let's boost!

Machine Learning with Tree-Based Models in R

Preparing Video For Download...