Principal Component Analysis Cheat Sheet
A cheat sheet for Principal Component Analysis covering scikit-learn implementation, explained variance, choosing component counts, and reconstruction error.
2 PagesIntermediateMar 12, 2026
PCA with scikit-learn
Reduce dimensionality and inspect explained variance.
python
from sklearn.decomposition import PCAfrom sklearn.preprocessing import StandardScalerX_scaled = StandardScaler().fit_transform(X) # PCA is scale-sensitivepca = PCA(n_components=2)X_pca = pca.fit_transform(X_scaled)print('Explained variance ratio:', pca.explained_variance_ratio_)print('Total variance captured:', pca.explained_variance_ratio_.sum())
Choosing the Number of Components
Use a scree plot or a variance target.
python
import numpy as npimport matplotlib.pyplot as pltpca_full = PCA().fit(X_scaled)cumulative = np.cumsum(pca_full.explained_variance_ratio_)plt.plot(cumulative)plt.xlabel('Number of components'); plt.ylabel('Cumulative explained variance')# Or let sklearn pick components that explain 95% of the variancepca_95 = PCA(n_components=0.95).fit(X_scaled)print(pca_95.n_components_)
Reconstruction
Project back to the original feature space.
python
X_reduced = pca.transform(X_scaled)X_reconstructed = pca.inverse_transform(X_reduced) # lossy reconstructionreconstruction_error = ((X_scaled - X_reconstructed) ** 2).mean()
Key Concepts
Core theory behind PCA.
- Principal components- Orthogonal directions of maximum variance in the data, ordered by how much variance they explain
- Eigenvectors/eigenvalues- Components are eigenvectors of the covariance matrix; eigenvalues indicate variance captured along each
- Explained variance ratio- Fraction of total dataset variance captured by each principal component
- Dimensionality reduction- Projecting onto the top-k components reduces feature count while preserving most information
- Standardization- Features must be scaled first, or high-variance features will dominate the components
- Whitening- whiten=True rescales components to unit variance, useful before some downstream algorithms
PCA via SVD (What sklearn Does Internally)
PCA is computed with singular value decomposition rather than eigendecomposition of the covariance matrix, for numerical stability.
python
import numpy as np# X_scaled is (n_samples, n_features), already centeredU, S, Vt = np.linalg.svd(X_scaled, full_matrices=False)# Principal components (loadings) are the rows of Vtcomponents = Vt[:2]# Scores (the projected data) equal U * S, and also X_scaled @ Vt.TX_pca_svd = U[:, :2] * S[:2]# Eigenvalues of the covariance matrix relate to singular values by:n = X_scaled.shape[0]explained_variance = (S ** 2) / (n - 1)explained_variance_ratio = explained_variance / explained_variance.sum()
IncrementalPCA for Large / Streaming Datasets
Fit PCA in mini-batches when the full dataset doesn't fit in memory.
python
from sklearn.decomposition import IncrementalPCAipca = IncrementalPCA(n_components=50, batch_size=500)for batch in np.array_split(X_scaled, len(X_scaled) // 500): ipca.partial_fit(batch)X_ipca = ipca.transform(X_scaled)# Randomized SVD solver is a faster approximate alternative for wide,# low-rank matrices (svd_solver='randomized' picked automatically# by PCA when n_components << min(n_samples, n_features))
Kernel PCA for Nonlinear Structure
Standard PCA only finds linear subspaces; Kernel PCA applies the kernel trick to capture nonlinear manifolds.
python
from sklearn.decomposition import KernelPCAkpca = KernelPCA(n_components=2, kernel='rbf', gamma=0.05, fit_inverse_transform=True)X_kpca = kpca.fit_transform(X_scaled)# Unlike linear PCA, Kernel PCA has no explained_variance_ratio_# because components live in an implicit, possibly infinite-dimensional space.# fit_inverse_transform=True enables an approximate pre-image reconstruction:X_approx = kpca.inverse_transform(X_kpca)
Inspecting Loadings and Feature Contributions
Rank original features by how strongly they load onto each principal component.
python
import pandas as pdloadings = pd.DataFrame( pca.components_.T, index=feature_names, columns=[f'PC{i+1}' for i in range(pca.n_components_)])# Top contributors to PC1, by absolute loading magnitudetop_pc1 = loadings['PC1'].abs().sort_values(ascending=False).head(10)# Squared loadings sum to 1 per component (they're unit vectors)assert np.allclose((loadings ** 2).sum(axis=0), 1.0)
Pitfalls & Caveats
Common mistakes that silently corrupt PCA results.
- Sign ambiguity- Eigenvectors are only defined up to sign; PC1 may point in opposite directions across re-fits or library versions — don't hardcode sign assumptions
- Fitting on train+test- Fit PCA only on the training split, then transform validation/test data with the fitted object, or you leak information across the split
- Outlier sensitivity- PCA maximizes variance, so a handful of extreme outliers can dominate the first component; consider RobustScaler or outlier removal first
- Categorical/one-hot features- PCA assumes continuous, roughly linear relationships; applying it directly to sparse one-hot columns usually produces uninterpretable components
- Variance ≠ relevance- The directions of highest variance aren't guaranteed to be predictive of your target label; PCA is unsupervised and label-blind
- Curse of correlated features- Near-duplicate or highly collinear features inflate the apparent variance explained by early components without adding real information
- n_components as float- Passing a float in (0,1) to n_components selects the minimum number of components needed to reach that cumulative explained variance
Pro Tip
PCA components are linear combinations of all original features, which makes them hard to interpret directly — inspect pca.components_ (the loadings) to see which original features contribute most to each principal component.
Was this cheat sheet helpful?
Explore Topics
#PrincipalComponentAnalysis#PrincipalComponentAnalysisCheatSheet#DataScience#Intermediate#PCAWithScikitLearn#ChoosingTheNumberOfComponents#Reconstruction#KeyConcepts#MachineLearning#CheatSheet#SkillVeris
