分類樹學習

Machine Learning with Tree-Based Models in Python

Elie Kawerk

Data Scientist

決策樹的基本元件

  • 決策樹（Decision-Tree）：由節點階層組成的資料結構。

  • 節點（Node）：提出問題或做出預測。

Machine Learning with Tree-Based Models in Python

決策樹的基本元件

三種節點：

  • 根節點（Root）：沒有父節點，提出問題並產生兩個子節點。

  • 內部節點（Internal node）：一個父節點，提出問題並產生兩個子節點。

  • 葉節點（Leaf）：一個父節點，沒有子節點 → 預測。

Machine Learning with Tree-Based Models in Python

預測

決策樹（已標註）

Machine Learning with Tree-Based Models in Python

Information Gain（IG）

資訊增益示意圖

Machine Learning with Tree-Based Models in Python

Information Gain（IG）

IG 公式

衡量節點不純度的準則 $I (node)$：

  • Gini 指數，
  • 熵（entropy）。 ...
Machine Learning with Tree-Based Models in Python

分類樹學習

  • 節點以遞迴方式成長。

  • 在每個節點，依據以下來切分資料：

    • 選擇特徵 $f$ 與切分點 $sp$，使 $IG(\text{node})$ 最大化。

  • 若 $IG (\text{node})$ = 0，將該節點標記為葉節點。

    ...

Machine Learning with Tree-Based Models in Python
# Import DecisionTreeClassifier
from sklearn.tree import DecisionTreeClassifier
# Import train_test_split
from sklearn.model_selection import train_test_split
# Import accuracy_score
from sklearn.metrics import accuracy_score
# Split dataset into 80% train, 20% test
X_train, X_test, y_train, y_test= train_test_split(X, y, 
                                                   test_size=0.2, 
                                                   stratify=y,
                                                   random_state=1)
# Instantiate dt, set 'criterion' to 'gini'
dt = DecisionTreeClassifier(criterion='gini', random_state=1)
Machine Learning with Tree-Based Models in Python

scikit-learn 的資訊準則

# Fit dt to the training set
dt.fit(X_train,y_train)

# Predict test-set labels
y_pred= dt.predict(X_test)

# Evaluate test-set accuracy
accuracy_score(y_test, y_pred)

0.92105263157894735
Machine Learning with Tree-Based Models in Python

一起來練習吧！

Machine Learning with Tree-Based Models in Python

Preparing Video For Download...