Data Versioning with DVC — Data Analysis & Feature Engineering

Machine-learning reproducibility requires not just versioning code but also versioning the data and models that code operates on, since the same code on different data produces different results and the meaning of a code version is incomplete without its corresponding data version. DVC, Data Version Control, is the tool that adds data and model versioning to Git-based workflows, tracking which data produced which model without storing large files in Git itself. It exists because datasets and trained models can be gigabytes or terabytes — impractical to store in Git — yet the question 'which data produced this model?' is as important for reproducibility as 'which code produced this model?'. Mastering DVC alongside Git creates the complete, reproducible ML audit trail that research integrity, regulatory compliance, and production reliability all demand.

Analogy🏏Cricket

🏏 Think of it like cricket: Reducing twenty batting statistics to two dimensions with PCA produces a flat projection that may look like one undifferentiated cloud, while t-SNE or UMAP finds that the data actually organises into distinct clusters — aggressive pinch-hitters, steady anchors, explosive finishers — whose separation PCA's flat projection smeared together. Just as the non-linear reduction reveals the natural groupings that flat projection could not, t-SNE and UMAP reveal structure that PCA misses because it can only flatten, not curve to follow the data's natural shape. The insight is that non-linear reduction methods follow the data's true curved geometry rather than forcing a flat projection, revealing the cluster structure and local neighbourhoods that linear methods like PCA cannot preserve.