Machine Learning with Tree-Based Models in Python
Elie Kawerk
Data Scientist
容易理解。
容易解釋。
易於使用。
彈性高:可描述非線性關係。
前處理少:不需標準化或正規化特徵,…
分類:只能產生正交決策邊界。
對訓練集的微小變動很敏感。
高變異:未受限制的 CART 易對訓練集過度擬合。
解法:集成式學習。
在同一資料集上訓練多個模型。
讓每個模型各自做預測。
中介模型:彙整各模型的預測。
最終預測:更穩健、較不易出錯。
最佳效果:模型各有所長且互補。

二元分類任務。
有 $N$ 個分類器輸出預測:$P_1$、$P_2$、…、$P_N$,其中 $P_i$ = 0 或 1。
中介模型預測:硬投票。

# Import functions to compute accuracy and split data
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
# Import models, including VotingClassifier meta-model
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.neighbors import KNeighborsClassifier as KNN
from sklearn.ensemble import VotingClassifier
# Set seed for reproducibility
SEED = 1
# Split data into 70% train and 30% test X_train, X_test, y_train, y_test = train_test_split(X, y, test_size= 0.3, random_state= SEED) # Instantiate individual classifiers lr = LogisticRegression(random_state=SEED) knn = KNN() dt = DecisionTreeClassifier(random_state=SEED)# Define a list called classifier that contains the tuples (classifier_name, classifier) classifiers = [('Logistic Regression', lr), ('K Nearest Neighbours', knn), ('Classification Tree', dt)]
# Iterate over the defined list of tuples containing the classifiers
for clf_name, clf in classifiers:
#fit clf to the training set
clf.fit(X_train, y_train)
# Predict the labels of the test set
y_pred = clf.predict(X_test)
# Evaluate the accuracy of clf on the test set
print('{:s} : {:.3f}'.format(clf_name, accuracy_score(y_test, y_pred)))
Logistic Regression: 0.947
K Nearest Neighbours: 0.930
Classification Tree: 0.930
# Instantiate a VotingClassifier 'vc'
vc = VotingClassifier(estimators=classifiers)
# Fit 'vc' to the traing set and predict test set labels
vc.fit(X_train, y_train)
y_pred = vc.predict(X_test)
# Evaluate the test-set accuracy of 'vc'
print('Voting Classifier: {.3f}'.format(accuracy_score(y_test, y_pred)))
Voting Classifier: 0.953
Machine Learning with Tree-Based Models in Python