Data Drift
Data drift is a change in the statistical distribution of a machine learning model's input data over time, which can degrade model performance even if the underlying relationship between inputs and outputs has not changed.
Definition
Data drift is a change in the statistical distribution of a machine learning model's input data over time, which can degrade model performance even if the underlying relationship between inputs and outputs has not changed.
Overview
Machine learning models assume, implicitly, that the data they see in production resembles the data they were trained on. Data drift happens when that assumption breaks down — new user demographics start using a product, a sensor is recalibrated and shifts its reading range, or a seasonal pattern shifts input distributions — so the model is now being asked to make predictions on inputs that look meaningfully different from what it learned on. There are a few common flavors of data drift. Covariate shift is when the distribution of input features changes but the relationship between inputs and the target stays the same. Label shift is when the distribution of the target outcome itself changes (for example, the overall fraud rate rising), even though the relationship given each input stays constant. Both are distinct from model drift, where the actual relationship between inputs and outputs changes — though in real systems these effects frequently overlap and can be difficult to tell apart without careful monitoring. Detecting data drift typically involves comparing statistical summaries — feature means, distributions, or more formal statistical tests — between the training data and recent production data, and flagging significant divergence. Because it doesn't require waiting for ground-truth labels, data drift monitoring can often catch problems earlier than model drift monitoring, which needs labeled outcomes to measure accuracy directly. Addressing drift may involve retraining, reweighting training data to better reflect the new distribution, or investigating upstream data pipeline changes that caused an unexpected shift — a common topic in MLOps & Model Deployment coursework.
Key Concepts
- Describes a shift in the statistical distribution of input data over time
- Includes covariate shift (input distribution changes) and label shift (outcome distribution changes)
- Distinct from model drift, where the input-output relationship itself changes
- Detectable without ground-truth labels, using statistical comparisons of feature distributions
- Often an early warning sign that precedes measurable accuracy degradation
- Addressed through retraining, data reweighting, or fixing upstream pipeline issues
Use Cases
Frequently Asked Questions
From the Blog
MLOps Explained: Deploy ML Models to Production
MLOps deploys ML models to production by combining DevOps practices with data and model versioning, automated pipelines, and drift monitoring.
Read More AI & TechnologyGreedy, Beam Search and Sampling: How Decoding Changes Output
Decoding is the step that turns a probability distribution into text, and it changes output more than most prompt edits do. This article compares greedy, beam search and stochastic sampling on determinism, diversity and factual drift, and names the task types each one suits.
Read More AI & TechnologyWhy Your RAG Pipeline Returns Irrelevant Chunks
Irrelevant retrieval has four common causes: an embedding mismatch between query and index, chunk boundaries that split the answer, vocabulary drift between how users ask and how documents phrase, and a filter or index setting quietly excluding the right document. Here is a check that isolates each.
Read More Cloud & CybersecurityTerraform drift detection and importing existing resources
Bring hand-made infrastructure under Terraform and keep it there: importing resources, reading refresh output and running scheduled drift checks.
Read More