Data Augmentation
Data augmentation is the practice of artificially expanding a training dataset by applying transformations to existing examples — such as rotating images or paraphrasing text — so a model sees more variation without needing to collect new…
Definition
Data augmentation is the practice of artificially expanding a training dataset by applying transformations to existing examples — such as rotating images or paraphrasing text — so a model sees more variation without needing to collect new labeled data.
Overview
Machine learning models, especially deep neural networks, tend to perform better with more training data, but collecting and labeling new data is often slow and expensive. Data augmentation offers a cheaper alternative: it generates new, plausible training examples by transforming the data a team already has, increasing effective dataset size and diversity while keeping labels valid. In computer vision, common augmentations include rotating, flipping, cropping, and color-jittering images, as well as more advanced techniques like mixing two images together or randomly erasing patches, all of which teach a model to be invariant to changes that shouldn't affect the correct label. In natural language processing, augmentation might involve synonym replacement, back-translation (translating text to another language and back), or paraphrasing with a generative model. In audio, common techniques include adding background noise, pitch shifting, and time stretching. Data augmentation is a core defense against overfitting, since a model trained on more varied examples is less likely to memorize quirks of a small dataset, and it is one of the standard techniques used to address class imbalance and AI bias by boosting the effective representation of underrepresented cases. It's closely related to but distinct from synthetic data, which generates entirely new examples rather than transforming existing ones. Augmentation is a standard part of the training pipeline in courses like PyTorch Deep Learning and TensorFlow & Keras.
Key Concepts
- Expands effective training data size by transforming existing labeled examples
- Common image techniques include rotation, flipping, cropping, and color jittering
- Common text techniques include synonym replacement, back-translation, and paraphrasing
- Common audio techniques include noise injection, pitch shifting, and time stretching
- Helps reduce overfitting by exposing models to more variation during training
- Used to address class imbalance by boosting underrepresented categories
- Distinct from synthetic data, which creates new examples rather than transforming real ones