Regularised linear regression addresses the high-variance problem that plain OLS faces when features are numerous, correlated, or weakly related to the target. By adding a penalty term to the OLS loss that grows with coefficient magnitude, regularisation shrinks coefficients toward zero, reducing variance at the cost of a small amount of bias — a trade that is almost always worthwhile when OLS overfits.
Ridge regression adds an L2 penalty (sum of squared coefficients), which shrinks all coefficients toward zero but never exactly to zero. Lasso adds an L1 penalty (sum of absolute coefficients), which produces exact zeros and performs automatic feature selection. ElasticNet combines both penalties, inheriting Ridge's ability to handle correlated features and Lasso's ability to zero out irrelevant ones. Together these three form the toolkit for regularised linear regression, covering the full spectrum from dense shrinkage to sparse selection.
Analogy🏏Cricket
🏏 Think of it like cricket: A batting coach who teaches only one shot — the forward defensive — gives advice with high bias: it is consistently wrong for deliveries that demand a drive or a pull, regardless of how much practice the batsman does. A coach who memorises every ball of the batsman's training career gives advice with high variance: he predicts each training ball perfectly but fails completely on new balls from a different bowler, because he learned the noise of that specific bowler rather than the underlying principles. The great coach finds the balance — teaching the core principles that generalise, without over-specifying for the particular training environment. Just as great coaching lies between the extremes, great ML models balance bias and variance.
🏏 Showing the Cricket analogy — a Cricket version isn’t available for this concept yet.