Stochastic Gradient Boosting (SGB)

Machine Learning dengan Model Berbasis Pohon di Python

Elie Kawerk

Data Scientist

Gradient Boosting: Kekurangan

GB melibatkan pencarian menyeluruh.
Setiap CART dilatih untuk menemukan titik belah dan fitur terbaik.
Dapat membuat CART memakai titik belah dan mungkin fitur yang sama.

Stochastic Gradient Boosting

Tiap pohon dilatih pada subset baris acak dari data latih.
Sampel instance (40%–80% dari data latih) diambil tanpa pengembalian.
Fitur disampel (tanpa pengembalian) saat memilih titik belah.
Hasil: keragaman ansambel meningkat.
Efek: menambah varians pada ansambel pohon.

Stochastic Gradient Boosting: Pelatihan

SGB

Stochastic Gradient Boosting di sklearn (dataset auto)

# Import models and utility functions
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error as MSE

# Set seed for reproducibility
SEED = 1

# Split dataset into 70% train and 30% test
X_train, X_test, y_train, y_test = train_test_split(X,y, 
                                                    test_size=0.3, 
                                                    random_state=SEED)

Stochastic Gradient Boosting di sklearn (dataset auto)

# Instantiate a stochastic GradientBoostingRegressor 'sgbt'
sgbt = GradientBoostingRegressor(max_depth=1, 
                                 subsample=0.8,
                                 max_features=0.2,
                                 n_estimators=300,             
                                 random_state=SEED)


# Fit 'sgbt' to the training set
sgbt.fit(X_train, y_train)

# Predict the test set labels
y_pred = sgbt.predict(X_test)

Stochastic Gradient Boosting di sklearn (dataset auto)

# Evaluate test set RMSE 'rmse_test'
rmse_test = MSE(y_test, y_pred)**(1/2)

# Print 'rmse_test'
print('Test set RMSE: {:.2f}'.format(rmse_test))

Test set RMSE: 3.95

Ayo berlatih!

Machine Learning dengan Model Berbasis Pohon di Python