Filter Methods — Correlation and Chi-Square — Data Analysis & Feature Engineering

Feature selection — choosing which variables to keep from a potentially large set — is as important as feature creation, because irrelevant and redundant features add noise, slow training, cause overfitting, and obscure the model's interpretability. Filter methods select features by evaluating a statistical property of each feature independently of any model, making them fast, model-agnostic, and easy to understand. They exist as the natural first pass because they catch obviously irrelevant or redundant features cheaply before any expensive model fitting, and they produce interpretable scores that help analysts understand which variables genuinely relate to the outcome. Mastering correlation-based and chi-square filter methods, and knowing their limits, is the foundation of principled feature selection that keeps models lean, fast, and generalising well.

Analogy🏏Cricket

🏏 Think of it like cricket: Reducing twenty batting statistics to two dimensions with PCA produces a flat projection that may look like one undifferentiated cloud, while t-SNE or UMAP finds that the data actually organises into distinct clusters — aggressive pinch-hitters, steady anchors, explosive finishers — whose separation PCA's flat projection smeared together. Just as the non-linear reduction reveals the natural groupings that flat projection could not, t-SNE and UMAP reveal structure that PCA misses because it can only flatten, not curve to follow the data's natural shape. The insight is that non-linear reduction methods follow the data's true curved geometry rather than forcing a flat projection, revealing the cluster structure and local neighbourhoods that linear methods like PCA cannot preserve.