Classification metrics

Validazione dei modelli in Python

Kasey Jones

Data Scientist

Classification metrics

  • Precision
  • Recall (also called sensitivity)
  • Accuracy
  • Specificity
  • F1-Score, and its variations
  • ...
Validazione dei modelli in Python

Classification metrics

  • Precision
  • Recall (also called sensitivity)
  • Accuracy
  • Specificity
  • F1-Score, and its variations
  • ...
Validazione dei modelli in Python

Confusion matrix

A confusion matrix is a 2x2 matrix that represents how many observations were predicted 0 or 1, and how many observations were actually 0 or 1. It can be used to calculate all kinds of accuracy metrics.

Validazione dei modelli in Python
from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_test, test_predictions)
print(cm)

array([[ 23,  7],
       [  8, 62]])

cm[<true_category_index>, <predicted_category_index>]
cm[1, 0]

8
Validazione dei modelli in Python

Accuracy

Accuracy can be calculated by using the main diagonal of the confusion matrix.

Validazione dei modelli in Python

Precision

Precision focuses on the observations that were predicted to be 1. Of all the predictions that were 1, how many of them actually were 1?

Validazione dei modelli in Python

Recall

Recall focuses on the observations that were actually 1’s. How many of these 1’s did we predict to be a 1?

Validazione dei modelli in Python

Accuracy, precision, recall

from sklearn.metrics import accuracy_score, precision_score, recall_score
accuracy_score(y_test, test_predictions)

.85

precision_score(y_test, test_predictions)

.8986

recall_score(y_test, test_predictions)

.8857
Validazione dei modelli in Python

Practice time

Validazione dei modelli in Python

Preparing Video For Download...