Performance evaluation

Python में Fraud Detection

Charlotte Werger

Data Scientist

Accuracy ही सब कुछ नहीं है

फ्रॉड डिटेक्शन में काम करते समय accuracy को नज़रअंदाज़ करें

False positives, false negatives, और पकड़ा गया वास्तविक फ्रॉड

Precision-recall tradeoff

परफॉर्मेंस मेट्रिक्स प्राप्त करना

# Import the packages
from sklearn.metrics import precision_recall_curve
from sklearn.metrics import average_precision_score

# Calculate average precision and the PR curve
average_precision = average_precision_score(y_test, predicted)

# Obtain precision and recall
precision, recall, _ = precision_recall_curve(y_test, predicted)

Precision-recall कर्व

एल्गोरिदम तुलना के लिए ROC कर्व

# Obtain model probabilities
probs = model.predict_proba(X_test)

# Print ROC_AUC score using probabilities 
print(metrics.roc_auc_score(y_test, probs[:, 1]))

0.9338879319822626

from sklearn.metrics import classification_report, confusion_matrix

# Obtain predictions 
predicted = model.predict(X_test)

# Print classification report using predictions
print(classification_report(y_test, predicted))

  precision    recall  f1-score   support

        0.0       0.99      1.00      1.00      2099
        1.0       0.96      0.80      0.87        91

avg / total       0.99      0.99      0.99      2190

# Print confusion matrix using predictions
print(confusion_matrix(y_test, predicted))

[[2096    3]
 [  18   73]]

अभ्यास करते हैं!

Python में Fraud Detection