ROC-AUC (Area Under the ROC Curve) condenses the entire ROC curve from the previous note into one single number — a widely used summary metric for comparing classifiers, independent of any specific threshold.
What the Area Represents
| AUC Value | Interpretation |
|---|---|
| 1.0 | A perfect classifier — the curve passes exactly through the top-left corner |
| 0.5 | No better than random guessing — the curve traces the diagonal |
| < 0.5 | Worse than random — though this typically means the model's predictions are systematically inverted, and simply flipping them would give AUC \(=1-\text{AUC}\) |
A Probabilistic Interpretation
ROC-AUC has a clean, intuitive meaning beyond just "area under a curve": it equals the probability that a randomly chosen actual-positive example receives a higher predicted score than a randomly chosen actual-negative example. An AUC of 0.9 means: pick one random true positive and one random true negative — 90% of the time, the model correctly ranks the positive example higher.
Numerical Example
Given scores for 2 positives (\([0.9, 0.6]\)) and 2 negatives (\([0.3, 0.7]\)), check every positive-negative pair: \((0.9, 0.3)\) — positive ranked higher, correct. \((0.9, 0.7)\) — positive ranked higher, correct. \((0.6, 0.3)\) — positive ranked higher, correct. \((0.6, 0.7)\) — negative ranked higher, incorrect. Out of 4 pairs, 3 are correctly ranked: \(\text{AUC} = \frac{3}{4}=0.75\).
Code
from sklearn.metrics import roc_auc_score
y_true = [1, 1, 0, 0]
y_scores = [0.9, 0.6, 0.3, 0.7]
print(roc_auc_score(y_true, y_scores)) # 0.75, matching the manual pairwise calculationCommon Mistakes
- Treating ROC-AUC as universally superior to other metrics for every dataset — on severely imbalanced data, PR-AUC (covered a few notes ahead) is often considered more informative, since ROC-AUC can look artificially high even when precision is genuinely poor.
- Forgetting that ROC-AUC evaluates ranking quality across all thresholds, not performance at any one specific threshold you'll actually deploy — a high AUC doesn't guarantee good performance at whatever single cutoff you end up choosing in production.
Interview Relevance
Q: "What does an ROC-AUC of 0.85 actually mean, in plain terms?" If you randomly picked one actual-positive example and one actual-negative example, the model would assign a higher predicted score to the positive example 85% of the time. It's a threshold-independent measure of how well the model ranks positives above negatives overall, not a statement about performance at any single specific cutoff.
Practice Question
For 1 positive example scored 0.4 and 2 negative examples scored 0.2 and 0.6, compute ROC-AUC by checking every positive-negative pair.
