Model evaluation: imbalanced classification models

Practicing Machine Learning Interview Questions in Python

Lisa Stuart

Data Scientist

Class imbalance

• Categorical target variable
  • Approx equal number observations/class
  • Large difference --> misleading results

Confusion matrix

¹ https://scaryscientist.blogspot.com/2016/03/confusion-matrix.html

Performance metrics

¹ https://scaryscientist.blogspot.com/2016/03/confusion-matrix.html

Metrics from the matrix

¹ https://scaryscientist.blogspot.com/2016/03/confusion-matrix.html

Resampling techniques

• Oversample minority class
• Undersample majority class
• NOTE: Split into test and train sets BEFORE re-sampling!

¹ https://www.svds.com/learning-imbalanced-classes/

Functions

Function	returns
sklearn.linear_model.LogisticRegression	logistic regression
sklearn.metrics.confusion_matrix(y_test,y_pred)	confusion matrix
sklearn.metrics.precision_score(y_test,y_pred)	precision
sklearn.metrics.recall_score(y_test,y_pred)	recall
sklearn.metrics.f1_score(y_test,y_pred)	f1 score
sklearn.utils.resample(deny, n_samples=len(approve))	resamples

Let's practice!

Practicing Machine Learning Interview Questions in Python