Feature Scaling — Standard, MinMax, Robust — Data Analysis & Feature Engineering

Feature scaling transforms numeric variables onto comparable ranges, a preprocessing step that is invisible in its effect on the data's information yet decisive for many algorithms. Scaling exists because numeric features often live on wildly different scales — runs in the hundreds, averages in the tens, strike rates above a hundred — and algorithms that measure distances or optimise with gradients treat large-magnitude features as more important purely because of their units, not their relevance. Without scaling, a distance-based model would be dominated by whichever feature happens to have the largest numbers, and gradient-based optimisation would converge slowly and unevenly. The three main scalers — standardisation, min-max normalisation, and robust scaling — each transform features differently and suit different situations, and knowing which to use, and which algorithms even need scaling, is essential to building models that treat features on their merits.

Analogy🏏Cricket

🏏 Think of it like cricket: Reducing twenty batting statistics to two dimensions with PCA produces a flat projection that may look like one undifferentiated cloud, while t-SNE or UMAP finds that the data actually organises into distinct clusters — aggressive pinch-hitters, steady anchors, explosive finishers — whose separation PCA's flat projection smeared together. Just as the non-linear reduction reveals the natural groupings that flat projection could not, t-SNE and UMAP reveal structure that PCA misses because it can only flatten, not curve to follow the data's natural shape. The insight is that non-linear reduction methods follow the data's true curved geometry rather than forcing a flat projection, revealing the cluster structure and local neighbourhoods that linear methods like PCA cannot preserve.