Model discrimination and impact

Credit Risk Modeling in Python

Michael Crabtree

Data Scientist, Ford Motor Company

Confusion matrices

  • Shows the number of correct and incorrect predictions for each loan_status

Confusion matrix with formulas

Credit Risk Modeling in Python

Default recall for loan status

  • Default recall (or sensitivity) is the proportion of true defaults predicted

Example classification report with default recall

Formula for default recall

Credit Risk Modeling in Python

Recall portfolio impact

  • Classification report - Underperforming Logistic Regression model

Example classification report with loan status highlights

Credit Risk Modeling in Python

Recall portfolio impact

  • Classification report - Underperforming Logistic Regression model

Example classification report with loan status highlights

  • Number of true defaults: 50,000
Loan Amount Defaults Predicted / Not Predicted Estimated Loss on Defaults
$50 .04 / .96 (50000 x .96) x 50 = $2,400,000
Credit Risk Modeling in Python

Recall, precision, and accuracy

  • Difficult to maximize all of them because there is a trade-off

Graph of non-default recall with default recall and accuracy

Credit Risk Modeling in Python

Let's practice!

Credit Risk Modeling in Python

Preparing Video For Download...