PCA — Principal Component Analysis

Principal Component Analysis is the workhorse of dimensionality reduction, creating a compact set of uncorrelated summary variables that capture as much of the original data's variance as possible. PCA exists because high-dimensional data is costly to model, harder to visualise, and plagued by the curse of dimensionality where distances become meaningless, and because many datasets have highly correlated features that contain redundant information that PCA condenses. Rather than selecting a subset of original features, PCA creates new composite features — principal components — that are linear combinations of the originals, each capturing a progressively smaller slice of the total variance. Understanding PCA means understanding both its power, compressing many correlated features into a few informative dimensions, and its cost, trading interpretability for compression, since the new components are abstract combinations rather than named variables.