Model Metrics

Marketing Analytics: Predicting Customer Churn in Python

Mark Peterson

Director of Data Science, Infoblox

Imbalanced classes

telco['Churn'].value_counts()

no     2850
yes     483
Name: Churn, dtype: int64

• Accuracy not a very useful metric

Precision

Metric	Formula
Precision	True Positives / (True Positives + False Positives)

 

• A model with high precision indicates:
  • Few false positives ("false alarms")
  • Not many non-churners were classified as churners

Recall

Metric	Formula
Recall/Sensitivity	True Positives / (True Positives + False Negatives)

 

• A model with high recall indicates that it correctly classified most churners

Precision vs. Recall

Confusion Matrix in scikit-learn

from sklearn.metrics import confusion_matrix

cm = confusion_matrix(y_test, y_pred)

Let's practice!

Marketing Analytics: Predicting Customer Churn in Python