PR-AUC (Area Under the Precision-Recall Curve) condenses the PR curve into a single number — the imbalanced-data-friendly counterpart to ROC-AUC.
Interpreting the Value
| PR-AUC Value | Interpretation |
|---|---|
| 1.0 | Perfect precision maintained across every recall level — an ideal classifier |
| Equal to the positive class's base rate (e.g. 0.01 for a 1%-positive dataset) | No better than a random classifier that predicts positive at the same rate as the true class balance |
Notice the crucial difference from ROC-AUC's baseline: a random classifier's ROC-AUC is always 0.5, regardless of class balance, but a random classifier's PR-AUC baseline depends on the positive class rate — for a severely imbalanced dataset, even a fairly "good-looking" PR-AUC needs to be compared against this much lower, imbalance-specific baseline to mean anything.
Why This Matters for Comparing PR-AUC Across Datasets
A PR-AUC of 0.3 might be excellent on a dataset with only 1% positive examples (far above the random baseline of 0.01) but mediocre on a dataset with 40% positive examples (below a baseline of 0.4) — PR-AUC values are not directly comparable across datasets with different class balances without accounting for this baseline shift.
Code
from sklearn.metrics import average_precision_score
import numpy as np
y_true = np.array([1,1,1,1,0,0,0,0])
y_scores = np.array([0.9, 0.8, 0.6, 0.4, 0.7, 0.3, 0.2, 0.1])
pr_auc = average_precision_score(y_true, y_scores)
print(pr_auc)Note that scikit-learn's average_precision_score is the standard, commonly used way to compute PR-AUC — it's a slightly different (and generally preferred) numerical integration than a naive trapezoidal rule applied directly to the precision-recall curve, but serves the same purpose.
ROC-AUC vs PR-AUC — When to Use Which
| ROC-AUC | PR-AUC | |
|---|---|---|
| Random baseline | Always 0.5, regardless of class balance | Equals the positive class rate — varies by dataset |
| Best suited for | Roughly balanced classes | Imbalanced classes, especially when the positive (minority) class is what matters most |
| Sensitivity to true negatives | High (FPR's denominator includes TN) | None (precision's denominator never includes TN) |
Common Mistakes
- Comparing PR-AUC values across datasets with meaningfully different class balances without adjusting for each dataset's own random baseline — a "high" PR-AUC on one dataset might actually be worse, relative to chance, than a "lower" PR-AUC on another.
- Reporting only ROC-AUC for a project involving a rare, high-stakes positive class — always check PR-AUC alongside it for imbalanced problems.
Interview Relevance
Q: "Why doesn't PR-AUC have a fixed baseline of 0.5, the way ROC-AUC does?" A random classifier's PR-AUC baseline equals the positive class's base rate in the dataset — for a 1%-positive dataset, random guessing achieves a PR-AUC around 0.01, not 0.5. This is because precision's denominator (\(TP+FP\)) directly reflects the actual class balance among positive predictions, unlike ROC's FPR, which is normalized in a way that keeps its random baseline fixed regardless of class balance.
Key Takeaways — Classification Metrics (So Far)
- The confusion matrix (TP, TN, FP, FN) is the foundation every classification metric in this category builds from.
- Accuracy is easily misleading on imbalanced data; precision and recall answer different questions and trade off against each other; F1 combines them via a harmonic mean that penalizes imbalance between the two.
- ROC/ROC-AUC and Precision-Recall/PR-AUC both summarize threshold-independent performance, but PR-based metrics are the more honest choice specifically for imbalanced datasets.
Practice Question
A dataset has 5% positive examples. What PR-AUC would a random classifier be expected to achieve, roughly?
