集成式學習

Machine Learning with Tree-Based Models in Python

Elie Kawerk

Data Scientist

CART 的優點

  • 容易理解。

  • 容易解釋。

  • 易於使用。

  • 彈性高:可描述非線性關係。

  • 前處理少:不需標準化或正規化特徵,…

Machine Learning with Tree-Based Models in Python

CART 的限制

  • 分類:只能產生正交決策邊界。

  • 對訓練集的微小變動很敏感。

  • 高變異:未受限制的 CART 易對訓練集過度擬合。

  • 解法:集成式學習。

Machine Learning with Tree-Based Models in Python

集成式學習

  • 在同一資料集上訓練多個模型。

  • 讓每個模型各自做預測。

  • 中介模型:彙整各模型的預測。

  • 最終預測:更穩健、較不易出錯。

  • 最佳效果:模型各有所長且互補。

Machine Learning with Tree-Based Models in Python

集成式學習:視覺化說明

集成式學習示意

Machine Learning with Tree-Based Models in Python

集成式學習實務:Voting Classifier

  • 二元分類任務。

  • 有 $N$ 個分類器輸出預測:$P_1$、$P_2$、…、$P_N$,其中 $P_i$ = 0 或 1。

  • 中介模型預測:硬投票。

Machine Learning with Tree-Based Models in Python

硬投票(Hard Voting)

硬投票

Machine Learning with Tree-Based Models in Python

sklearn 的 Voting Classifier(Breast-Cancer 資料集)

# Import functions to compute accuracy and split data
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split

# Import models, including VotingClassifier meta-model
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.neighbors import KNeighborsClassifier as KNN
from sklearn.ensemble import VotingClassifier

# Set seed for reproducibility
SEED = 1
Machine Learning with Tree-Based Models in Python

sklearn 的 Voting Classifier(Breast-Cancer 資料集)

# Split data into 70% train and 30% test
X_train, X_test, y_train, y_test = train_test_split(X, y,
                                                    test_size= 0.3,
                                                    random_state= SEED)
# Instantiate individual classifiers
lr = LogisticRegression(random_state=SEED)
knn = KNN()
dt = DecisionTreeClassifier(random_state=SEED)

# Define a list called classifier that contains the tuples (classifier_name, classifier) classifiers = [('Logistic Regression', lr), ('K Nearest Neighbours', knn), ('Classification Tree', dt)]
Machine Learning with Tree-Based Models in Python
# Iterate over the defined list of tuples containing the classifiers
for clf_name, clf in classifiers:
    #fit clf to the training set
    clf.fit(X_train, y_train)

    # Predict the labels of the test set
    y_pred = clf.predict(X_test)

    # Evaluate the accuracy of clf on the test set
    print('{:s} : {:.3f}'.format(clf_name, accuracy_score(y_test, y_pred)))
Logistic Regression: 0.947
K Nearest Neighbours: 0.930
Classification Tree: 0.930
Machine Learning with Tree-Based Models in Python

sklearn 的 Voting Classifier(Breast-Cancer 資料集)

# Instantiate a VotingClassifier 'vc'
vc = VotingClassifier(estimators=classifiers) 

# Fit 'vc' to the traing set and predict test set labels
vc.fit(X_train, y_train)   
y_pred = vc.predict(X_test)

# Evaluate the test-set accuracy of 'vc'
print('Voting Classifier: {.3f}'.format(accuracy_score(y_test, y_pred)))
Voting Classifier: 0.953
Machine Learning with Tree-Based Models in Python

一起來練習吧!

Machine Learning with Tree-Based Models in Python

Preparing Video For Download...