Gradient boosting

Metode Ensemble di Python

Román de las Heras

Data Scientist, Appodeal

Pengantar gradient boosting machine

fungsi_objektif.png

  1. Model awal (weak estimator): $y\sim f_1(X)$
  2. Model baru memodelkan residual: $y-f_1(X)\sim f_2(X)$
  3. Model aditif baru: $y\sim f_1(X)+f_2(X)$
  4. Ulangi $n$ kali atau sampai galat cukup kecil
  5. Model aditif akhir: $$y\sim f_1(X)+f_2(X)+ ... +f_n(x)=\sum_{i=1}^{n} f_i(X)$$
Metode Ensemble di Python

Ekuivalen dengan gradient descent

fungsi_residual.png

Gradient Descent:

fungsi_loss.png

fungsi_gradien.png

Residual = Gradien Negatif

fungsi_gradien_negatif.png

Metode Ensemble di Python

Gradient boosting classifier

Gradient Boosting Classifier

from sklearn.ensemble import GradientBoostingClassifier

clf_gbm = GradientBoostingClassifier(
   n_estimators=100,
   learning_rate=0.1,
   max_depth=3,
   min_samples_split,
   min_samples_leaf,
   max_features
)
  • n_estimators
    • Default: 100
  • learning_rate
    • Default: 0.1
  • max_depth
    • Default: 3
  • min_samples_split
  • min_samples_leaf
  • max_features
Metode Ensemble di Python

Gradient boosting regressor

Gradient Boosting Regressor

from sklearn.ensemble import GradientBoostingRegressor

reg_gbm = GradientBoostingRegressor(
   n_estimators=100,
   learning_rate=0.1,
   max_depth=3,
   min_samples_split,
   min_samples_leaf,
   max_features
)
Metode Ensemble di Python

Saatnya boosting!

Metode Ensemble di Python

Preparing Video For Download...