Learn to measure model accuracy with essential regression metrics. We break down RMSE, MAE, and R-squared so you can evaluate your predictions like a pro.
Previously in this course, we covered training the baseline linear model. Now that you have a model spitting out predictions, you need a way to determine if those numbers are actually good. In this lesson, we move beyond "it looks okay" and into the quantitative world of regression metrics.
When predicting continuous values like house prices or temperature, we measure performance by calculating the distance between our predicted values ($\hat{y}$) and the actual ground truth values ($y$).
There is no "perfect" metric. A single number often hides the nature of your model's errors. We use a combination of metrics to understand the magnitude, direction, and explanatory power of our predictions.
MAE is the simplest metric: it is the average of the absolute differences between predictions and actuals. If your model predicts a house price is $500k but it's actually $520k, the error is $20k.
Mathematically, it's the mean of $|y - \hat{y}|$. Because it uses absolute values, it doesn't penalize large errors more than small ones. It tells you, on average, how many units "off" your model is.
RMSE is the standard for most regression tasks. We take the difference, square it, find the average, and then take the square root.
The squaring step is critical: it heavily penalizes large errors. If your model is consistently off by a small amount, your RMSE will be low. If your model is mostly accurate but makes a few massive "outlier" mistakes, your RMSE will spike.
While MAE and RMSE tell you about the scale of your error, $R^2$ tells you about the quality of your fit. Often called the "Coefficient of Determination," it represents the proportion of variance in the target variable that your model explains.
An $R^2$ of 1.0 is a perfect model; an $R^2$ of 0.0 is equivalent to a model that simply guesses the average value of the target every single time.
In practice, you shouldn't calculate these by hand. Use scikit-learn to ensure precision and consistency.
PYTHONimport numpy as np from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score # Simulated data from our project y_true = np.array([100, 150, 200, 250, 300]) y_pred = np.array([105, 145, 210, 240, 310]) # Calculate metrics mae = mean_absolute_error(y_true, y_pred) rmse = np.sqrt(mean_squared_error(y_true, y_pred)) r2 = r2_score(y_true, y_pred) print(f"MAE: {mae:.2f}") print(f"RMSE: {rmse:.2f}") print(f"R-squared: {r2:.2f}")
In this example, the MAE tells us we are off by about 7 units on average. The RMSE is slightly higher, which alerts us that our errors are spread out.
Using the dataset you cleaned in project dataset initialization, load your test set predictions from your previous model.
Regression evaluation is about understanding the "distance" between your predictions and reality. Use MAE for an intuitive look at average error, RMSE to catch and penalize large mistakes, and R-squared to judge the overall explanatory power of your model. By mastering these regression metrics, you gain the ability to diagnose model performance objectively.
Up next: We will dive into the confusion matrix to see how we evaluate models that categorize data rather than predicting continuous numbers.
Learn why high training performance often masks poor real-world results. Discover how to compare training and testing error to master model generalization.
Read moreLearn how to evaluate model calibration using calibration curves and the Brier score. Ensure your predicted probabilities are accurate representations of reality.
Regression Evaluation Metrics