This note collects the standard evaluation metrics for regression tasks — MAE, MSE and RMSE were already introduced as training losses in the Loss Functions category; here, the focus shifts to R², the metric most often reported alongside them for evaluating how good a regression model actually is.
MAE, MSE, RMSE — As Evaluation Metrics
Any of the regression losses covered in Mean Absolute Error, Mean Squared Error, and Root Mean Squared Error can double as an evaluation metric, computed on validation or test data rather than used to drive gradient updates. RMSE is the most commonly reported of the three specifically because its units match the target variable directly, as covered in that earlier note.
R² (Coefficient of Determination)
\(SS_{\text{res}}\) (residual sum of squares) is exactly the numerator behind MSE — the model's actual prediction error. \(SS_{\text{tot}}\) (total sum of squares) measures how much the true values vary around their own mean \(\bar y\) — essentially, the error a trivial "always predict the average" baseline model would make. \(R^2\) measures what fraction of that baseline variance the model actually explains.
Interpreting R²
| \(R^2\) Value | Interpretation |
|---|---|
| 1.0 | The model perfectly predicts every value — \(SS_{\text{res}}=0\) |
| 0.0 | The model performs exactly as well as always predicting the mean — no better than a trivial baseline |
| Negative | The model performs worse than simply always predicting the mean — a genuinely bad model |
Numerical Example
True values \([10, 20, 30]\), predictions \([12, 18, 35]\), mean \(\bar y = 20\). \(SS_{\text{res}} = (10-12)^2+(20-18)^2+(30-35)^2 = 4+4+25=33\). \(SS_{\text{tot}} = (10-20)^2+(20-20)^2+(30-20)^2 = 100+0+100=200\).
The model explains about 83.5% of the variance in the true values relative to simply always guessing the mean.
Code
from sklearn.metrics import r2_score, mean_squared_error
import numpy as np
y_true = np.array([10, 20, 30])
y_pred = np.array([12, 18, 35])
print(r2_score(y_true, y_pred)) # 0.835
print(np.sqrt(mean_squared_error(y_true, y_pred))) # RMSE, for comparisonCommon Mistakes
- Assuming R² is always between 0 and 1 — a genuinely poor model (worse than the trivial mean-prediction baseline) can produce a negative R², which is a valid, meaningful result signaling a serious problem.
- Comparing R² values across datasets with very different amounts of inherent variance — a high R² on a low-variance dataset and a lower R² on a high-variance, noisier dataset don't necessarily reflect a real difference in model quality.
Interview Relevance
Q: "What does it mean for a regression model to have a negative R²?" It means the model performs worse than the trivial baseline of always predicting the target's mean value — a clear sign of a poorly fit or fundamentally broken model, since even ignoring the input features entirely and guessing the average would do better.
Practice Question
For true values \([5, 10, 15]\) and predictions \([5, 10, 15]\) (a perfect model), what is \(R^2\)? What about for predictions that are all exactly the mean of the true values, \([10, 10, 10]\)?
