Polynomial regression extends linear regression to capture curved (non-linear) relationships by adding powers of the original features as new predictors, then fitting a standard linear model on the augmented feature matrix. The model remains linear in its parameters — the fitting machinery is unchanged — but it can trace curves, humps, and S-shapes that a straight line cannot.
This technique exists because many real relationships are non-linear: performance improves with practice then plateaus, risk rises sharply at extremes, and dose-response curves are typically sigmoidal. Polynomial regression is the simplest way to capture these patterns within the interpretable linear modelling framework. Its central challenge is degree selection: too low a degree underfits (straight-line limitation), too high a degree overfits (oscillating wildly through the training points), and finding the right degree requires cross-validation.
Analogy🏏Cricket
🏏 Think of it like cricket: A batting coach who teaches only one shot — the forward defensive — gives advice with high bias: it is consistently wrong for deliveries that demand a drive or a pull, regardless of how much practice the batsman does. A coach who memorises every ball of the batsman's training career gives advice with high variance: he predicts each training ball perfectly but fails completely on new balls from a different bowler, because he learned the noise of that specific bowler rather than the underlying principles. The great coach finds the balance — teaching the core principles that generalise, without over-specifying for the particular training environment. Just as great coaching lies between the extremes, great ML models balance bias and variance.
🏏 Showing the Cricket analogy — a Cricket version isn’t available for this concept yet.