Cross-Validation
Cross-validation is a model evaluation technique that repeatedly splits a dataset into different training and validation subsets, trains and tests the model on each split, and averages the results to produce a more reliable estimate of how…
Definition
Cross-validation is a model evaluation technique that repeatedly splits a dataset into different training and validation subsets, trains and tests the model on each split, and averages the results to produce a more reliable estimate of how the model will perform on unseen data.
Overview
A single train/test split can give a misleading picture of model performance — if the test set happens to contain unusually easy or unusually hard examples by chance, the reported accuracy might not represent how the model will actually perform in general. Cross-validation addresses this by using multiple different splits and averaging the results, smoothing out the effect of any one particular split being unrepresentative. The most common approach is k-fold cross-validation: the dataset is divided into k equal-sized folds, and the model is trained k times, each time using k-1 folds for training and the remaining fold for validation, rotating which fold is held out each time. The final performance estimate is the average across all k runs, and the variation across runs also gives a sense of how stable the model's performance is. Stratified k-fold is a common variant that preserves the class distribution in each fold, which matters for imbalanced classification problems. For time-series data, a specialized time-series split is used instead, since randomly shuffling folds would leak future information into training, which the model wouldn't have access to at prediction time in the real world. Cross-validation is especially important during hyperparameter tuning, where many candidate configurations are compared against each other — without cross-validation, a team risks selecting hyperparameters that happen to fit one particular validation split well rather than generalizing broadly, a subtle form of overfitting to the validation set itself. It's one of the first practical techniques taught in applied ML coursework, including Machine Learning Fundamentals and Python for AI & ML.
Key Concepts
- Produces a more reliable performance estimate than a single train/test split
- K-fold cross-validation trains and validates the model k times across rotating folds
- Stratified k-fold preserves class distribution across folds for imbalanced data
- Time-series cross-validation avoids leaking future data into training splits
- Helps detect overfitting to a specific validation split during hyperparameter tuning
- Provides both an average performance estimate and a measure of result stability
Use Cases
Frequently Asked Questions
From the Blog
What Is Cross-Validation in Machine Learning
Cross-validation tests a model on multiple data splits instead of one, giving a reliable estimate of how it will perform on unseen data. Here is how it works.
Read More AI & TechnologyHow to Split Train, Validation and Held-Out Sets Properly
Split fine-tuning data by grouping related examples before you split, deduplicating near-identical text across the boundary, and reserving a held-out set that is never used for any decision. This covers leakage sources specific to text data, how to detect them, and what an untouched final set is actually for.
Read More AI & TechnologyReranking in RAG: When a Cross-Encoder Earns Its Latency
A cross-encoder reranker earns its latency when your first-stage retriever has high recall at a wide k but poor ordering in the top few. This article explains the two-stage pattern, the recall condition that makes reranking worthwhile, and how to measure whether it is paying for itself.
Read More AI & TechnologyCross-modal search: retrieving images and audio with text queries
Shared embedding spaces let text queries retrieve images and audio. Learn how the spaces are built, where gist-level matching fails, and how to evaluate results.
Read More