Regression Diagnostics and Residual Plots — Machine Learning with Scikit-learn

Summary metrics like RMSE and R2 collapse the model's performance across all predictions into single numbers, hiding the patterns in how the model fails. Regression diagnostics — residual plots, Q-Q plots, and influence diagnostics — reveal these hidden patterns and are essential for confirming that a linear model's assumptions hold and for identifying the specific predictions where the model breaks down. A model that looks adequate by summary metrics can systematically fail for high-value predictions, for a specific subgroup, or for predictions at the extremes of the feature space, and these failures are invisible without diagnostic plots.

This lesson covers the standard diagnostic toolkit: the residual-versus-fitted plot for detecting non-linearity and heteroscedasticity, the scale-location plot for confirming homoscedasticity, the Q-Q plot for checking residual normality, the residual-versus-leverage plot for identifying influential outliers, and partial regression plots for assessing individual feature relationships. Together these five plots constitute the standard diagnostic suite for any deployed linear regression model.

Analogy🏏Cricket

🏏 Think of it like cricket: A batting coach who teaches only one shot — the forward defensive — gives advice with high bias: it is consistently wrong for deliveries that demand a drive or a pull, regardless of how much practice the batsman does. A coach who memorises every ball of the batsman's training career gives advice with high variance: he predicts each training ball perfectly but fails completely on new balls from a different bowler, because he learned the noise of that specific bowler rather than the underlying principles. The great coach finds the balance — teaching the core principles that generalise, without over-specifying for the particular training environment. Just as great coaching lies between the extremes, great ML models balance bias and variance.