隨機梯度提升（SGB）

Machine Learning with Tree-Based Models in Python

Elie Kawerk

Data Scientist

梯度提升：缺點

  • GB 需要進行全面的搜尋程序。

  • 每個 CART 都會訓練以找出最佳分割點與特徵。

  • 可能導致多棵 CART 使用相同的分割點，甚至相同的特徵。

Machine Learning with Tree-Based Models in Python

隨機梯度提升

  • 每棵樹都在訓練資料的隨機子集（列）上訓練。

  • 抽樣實例為訓練集的 40%-80%，且為不放回抽樣。

  • 選擇分割點時，特徵也以不放回方式抽樣。

  • 結果：進一步提升集成的多樣性。

  • 影響：為樹的集成再加入變異。

Machine Learning with Tree-Based Models in Python

隨機梯度提升：訓練

SGB

Machine Learning with Tree-Based Models in Python

sklearn 的隨機梯度提升（auto 資料集）

# Import models and utility functions
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error as MSE

# Set seed for reproducibility
SEED = 1

# Split dataset into 70% train and 30% test
X_train, X_test, y_train, y_test = train_test_split(X,y, 
                                                    test_size=0.3, 
                                                    random_state=SEED)
Machine Learning with Tree-Based Models in Python

sklearn 的隨機梯度提升（auto 資料集）

# Instantiate a stochastic GradientBoostingRegressor 'sgbt'
sgbt = GradientBoostingRegressor(max_depth=1, 
                                 subsample=0.8,
                                 max_features=0.2,
                                 n_estimators=300,             
                                 random_state=SEED)


# Fit 'sgbt' to the training set
sgbt.fit(X_train, y_train)

# Predict the test set labels
y_pred = sgbt.predict(X_test)
Machine Learning with Tree-Based Models in Python

sklearn 的隨機梯度提升（auto 資料集）

# Evaluate test set RMSE 'rmse_test'
rmse_test = MSE(y_test, y_pred)**(1/2)

# Print 'rmse_test'
print('Test set RMSE: {:.2f}'.format(rmse_test))

Test set RMSE: 3.95
Machine Learning with Tree-Based Models in Python

一起來練習吧！

Machine Learning with Tree-Based Models in Python

Preparing Video For Download...