What are the ROC Curve and AUC?
Learn what the ROC curve and AUC measure, their probabilistic meaning, a scikit-learn example, and when precision-recall curves beat ROC on imbalanced data.
Expected Interview Answer
The ROC (Receiver Operating Characteristic) curve plots the true positive rate against the false positive rate across every possible classification threshold, and AUC (Area Under the Curve) summarises that whole curve as a single number between 0 and 1 measuring how well the model separates the two classes.
As you sweep the decision threshold from strict to lenient, the model catches more positives (TPR rises) but also raises more false alarms (FPR rises); the ROC curve traces this trade-off. A perfect classifier hugs the top-left corner with AUC = 1.0, while random guessing follows the diagonal with AUC = 0.5. AUC has a neat probabilistic meaning: it is the probability that the model scores a randomly chosen positive higher than a randomly chosen negative, which makes it threshold-independent and robust to class imbalance in ranking quality — though for highly imbalanced data the precision-recall curve is often more informative.
- Evaluates a model across all thresholds at once
- Threshold-independent single-number summary
- Intuitive probabilistic interpretation of ranking quality
- Enables fair comparison between competing models
- Helps choose an operating threshold from the curve
AI Mentor Explanation
A selector ranks players by predicted match impact. The ROC curve asks: as you lower the bar for selection, how many genuine match-winners do you pick up versus how many flops sneak in? AUC is the chance that a randomly chosen match-winner was rated above a randomly chosen flop. A great scout ranks winners above flops almost every time, pushing AUC toward one.
Step-by-Step Explanation
Step 1
Get predicted probabilities
Use the model's score or probability for the positive class rather than the hard 0/1 label.
Step 2
Sweep the threshold
For each possible cutoff from 0 to 1, convert scores into predictions and count TP, FP, TN, FN.
Step 3
Compute TPR and FPR
At each threshold calculate true positive rate = TP/(TP+FN) and false positive rate = FP/(FP+TN).
Step 4
Plot the curve
Plot TPR (y-axis) against FPR (x-axis); the resulting line is the ROC curve.
Step 5
Measure the area
Integrate under the curve to get AUC — 1.0 is perfect, 0.5 is random, below 0.5 is worse than chance.
What Interviewer Expects
- Correct axes: TPR versus FPR across thresholds
- The meaning of AUC = 1.0, 0.5 and < 0.5
- The probabilistic interpretation of AUC
- Why ROC-AUC is threshold-independent
- When PR curves are preferable to ROC on imbalanced data
Common Mistakes
- Plotting precision vs recall and calling it ROC
- Confusing FPR with precision
- Claiming AUC = 0.5 means a perfect model
- Passing hard labels instead of probability scores to roc_auc_score
- Ignoring that ROC can look optimistic on severely imbalanced data
Best Answer (HR Friendly)
“The ROC curve shows how well a model separates two classes as you make it stricter or more lenient, plotting the hits it catches against the false alarms it raises. AUC squeezes that whole picture into one score from 0.5 (random guessing) to 1.0 (perfect), so higher means better separation.”
Code Example
import numpy as np
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_curve, roc_auc_score
X, y = make_classification(n_samples=1000, weights=[0.7, 0.3], random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
model = LogisticRegression().fit(X_train, y_train)
# Use probabilities of the positive class, NOT hard labels
y_scores = model.predict_proba(X_test)[:, 1]
fpr, tpr, thresholds = roc_curve(y_test, y_scores)
auc = roc_auc_score(y_test, y_scores)
print(f"AUC: {auc:.3f}")
print(f"First few FPR: {np.round(fpr[:5], 3)}")
print(f"First few TPR: {np.round(tpr[:5], 3)}")Follow-up Questions
- Why can ROC-AUC look overly optimistic on highly imbalanced datasets?
- How does the precision-recall curve differ from the ROC curve?
- What does an AUC below 0.5 tell you about a model?
- How would you pick an operating threshold from an ROC curve?
- How does AUC relate to the Gini coefficient?
MCQ Practice
1. The ROC curve plots which two quantities?
ROC plots the true positive rate (y) against the false positive rate (x) across thresholds.
2. An AUC of 0.5 indicates a model that:
AUC = 0.5 corresponds to the diagonal — the model ranks positives and negatives no better than chance.
3. Which input should you pass to roc_auc_score for a probabilistic model?
AUC is computed from ranked scores, so you pass predicted probabilities, not thresholded labels.
Flash Cards
ROC curve axes — True positive rate (y) versus false positive rate (x) across all thresholds.
AUC = 1.0 vs 0.5 — 1.0 is a perfect separator; 0.5 is random guessing (the diagonal).
Probabilistic meaning of AUC — Probability a random positive is scored higher than a random negative.
ROC vs PR curve — PR curves are more informative than ROC on severely imbalanced datasets.