가방 밖 평가 (Out-of-Bag)

Python으로 배우는 트리 기반 Machine Learning

Elie Kawerk

Data Scientist

배깅

일부 인스턴스는 하나의 모델에 여러 번 샘플링될 수 있음,
일부 인스턴스는 전혀 샘플링되지 않을 수 있음.

가방 밖(OOB) 인스턴스

평균적으로 각 모델에 대해 학습 인스턴스의 63%가 샘플링됨.
나머지 37%는 OOB 인스턴스가 됨.

OOB 평가

oob-evaluation

sklearn의 OOB 평가 (유방암 데이터셋)

# Import models and split utility function
from sklearn.ensemble import BaggingClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split

# Set seed for reproducibility
SEED = 1

# Split data into 70% train and 30% test
X_train, X_test, y_train, y_test =  train_test_split(X, y, test_size= 0.3,
                                                     stratify= y,
                                                     random_state=SEED)

# Instantiate a classification-tree 'dt'
dt = DecisionTreeClassifier(max_depth=4,
                            min_samples_leaf=0.16,
                            random_state=SEED)

# Instantiate a BaggingClassifier 'bc'; set oob_score = True 
bc = BaggingClassifier(base_estimator=dt, n_estimators=300,
                       oob_score=True, n_jobs=-1)


# Fit 'bc' to the training set
bc.fit(X_train, y_train)

# Predict the test set labels
y_pred = bc.predict(X_test)

# Evaluate test set accuracy
test_accuracy = accuracy_score(y_test, y_pred)

# Extract the OOB accuracy from 'bc'
oob_accuracy = bc.oob_score_

# Print test set accuracy
print('Test set accuracy: {:.3f}'.format(test_accuracy))

Test set accuracy: 0.936

# Print OOB accuracy
print('OOB accuracy: {:.3f}'.format(oob_accuracy))

OOB accuracy: 0.925

Ayo berlatih!

Python으로 배우는 트리 기반 Machine Learning