ROC & AUC Cheat Sheet

How the ROC curve and AUC score measure binary classifier performance across thresholds, with plotting and interpretation using scikit-learn.

1 PageIntermediateMar 12, 2026

ROC Curve & AUC Score

Compute and plot the ROC curve.

python

from sklearn.metrics import roc_curve, roc_auc_score, RocCurveDisplayimport matplotlib.pyplot as plt# y_scores = predicted probabilities for the positive classy_scores = model.predict_proba(X_test)[:, 1]fpr, tpr, thresholds = roc_curve(y_test, y_scores)auc = roc_auc_score(y_test, y_scores)print(f"AUC: {auc:.3f}")RocCurveDisplay(fpr=fpr, tpr=tpr, roc_auc=auc).plot()plt.plot([0, 1], [0, 1], linestyle="--", label="Random classifier")plt.legend()plt.show()

Precision-Recall Curve

Better alternative for rare-positive-class problems.

python

from sklearn.metrics import precision_recall_curve, average_precision_score# Use PR curve instead of ROC when the positive class is rareprecision, recall, thresholds = precision_recall_curve(y_test, y_scores)ap = average_precision_score(y_test, y_scores)print(f"Average Precision: {ap:.3f}")

ROC Concepts

Key terms behind the ROC curve.

ROC curve- plots True Positive Rate (recall) vs False Positive Rate at every threshold
True Positive Rate (TPR)- TP / (TP + FN); same as recall/sensitivity
False Positive Rate (FPR)- FP / (FP + TN); fraction of negatives incorrectly flagged
AUC- area under the ROC curve; probability a random positive scores higher than a random negative
AUC = 0.5- no better than random guessing
AUC = 1.0- perfect separation between classes
Threshold- the cutoff probability used to convert scores into class predictions

Interpretation Tips

How to read and apply ROC/AUC in practice.

Threshold selection- pick the threshold on the curve closest to the top-left corner, or by business cost
Comparing models- a higher AUC means better ranking of positives above negatives on average
Imbalanced data caveat- ROC-AUC can look optimistic when negatives vastly outnumber positives
PR-AUC alternative- preferred over ROC-AUC for rare-event/imbalanced classification tasks
Not threshold-specific- AUC summarizes performance across all thresholds, not one operating point

Pro Tip

For heavily imbalanced datasets (e.g. fraud detection with 1% positives), prefer the Precision-Recall AUC over ROC-AUC — ROC-AUC can stay deceptively high because the large number of true negatives dominates the false positive rate.

Was this cheat sheet helpful?

Confusion Matrix & Metrics Cheat Sheet

A/B Testing Cheat Sheet