What are Overfitting and Underfitting?
Learn what overfitting and underfitting mean in machine learning, how to detect them, their causes, and proven fixes like regularization and early stopping.
Expected Interview Answer
Overfitting is when a model learns the training data too well — including its noise — so it performs great on training data but poorly on unseen data; underfitting is when a model is too simple to capture the underlying pattern, so it performs poorly on both training and test data.
Overfitting shows up as a large gap between low training error and high validation error, and is caused by excessive model capacity, too many features, or too little data. Underfitting shows high error everywhere and comes from insufficient capacity or over-regularization. The goal is a model that generalizes: cures for overfitting include more data, regularization, dropout, early stopping and simpler models, while underfitting is fixed with a richer model, more features, or less regularization.
- Explains why a model fails on real-world data
- Guides model complexity and capacity choices
- Motivates train/validation/test splits
- Underpins regularization and early stopping
- Central to diagnosing the bias-variance tradeoff
AI Mentor Explanation
A batter who memorises the exact deliveries from one net session scores freely there but flounders in a real match against new bowlers — that is overfitting. A batter who never practises footwork and pokes at everything struggles in the nets and the match alike — that is underfitting. A generalising player learns transferable technique, not one bowler's tricks.
Step-by-Step Explanation
Step 1
Split your data
Hold out validation and test sets so you can measure generalization, not just training fit.
Step 2
Compare train vs validation error
A low train error with high validation error signals overfitting; high error on both signals underfitting.
Step 3
Diagnose capacity
Plot learning curves to see whether the model is too complex (overfit) or too simple (underfit).
Step 4
Fix overfitting
Add data, apply regularization (L1/L2), dropout, early stopping, or reduce model complexity.
Step 5
Fix underfitting
Increase model capacity, add or engineer features, train longer, or reduce regularization.
What Interviewer Expects
- Clear definitions of both terms
- Understanding of the train/validation error gap
- Knowledge of concrete remedies for each
- Link to generalization and the bias-variance tradeoff
- Ability to diagnose via learning curves
Common Mistakes
- Confusing overfitting with underfitting
- Thinking high training accuracy always means a good model
- Ignoring the validation set when tuning
- Believing more epochs or complexity is always better
- Not mentioning any concrete remedy
Best Answer (HR Friendly)
“Overfitting is when a model memorizes the training examples so it does great on them but fails on new data, while underfitting is when the model is too simple to learn the pattern and does poorly everywhere. The aim is a model in the middle that generalizes well to data it has never seen.”
Code Example
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
X, y = make_classification(n_samples=1000, n_features=20, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
# An unrestricted deep tree tends to overfit
deep = DecisionTreeClassifier(random_state=42)
deep.fit(X_train, y_train)
print('deep train:', deep.score(X_train, y_train)) # ~1.0
print('deep test :', deep.score(X_test, y_test)) # much lower -> overfit
# Limiting depth reduces variance and improves generalization
reg = DecisionTreeClassifier(max_depth=4, random_state=42)
reg.fit(X_train, y_train)
print('reg train:', reg.score(X_train, y_train))
print('reg test :', reg.score(X_test, y_test)) # gap is smallerFollow-up Questions
- How do learning curves help you distinguish overfitting from underfitting?
- What regularization techniques reduce overfitting?
- How does early stopping prevent overfitting?
- Why does more training data usually help overfitting but not underfitting?
- How does model capacity relate to the bias-variance tradeoff?
MCQ Practice
1. A model has 99% training accuracy but 65% test accuracy. What is happening?
A large gap where training accuracy far exceeds test accuracy is the classic signature of overfitting.
2. Which technique does NOT help combat overfitting?
Increasing model complexity generally increases variance and worsens overfitting; the others all reduce it.
3. Underfitting is typically caused by?
Underfitting means the model lacks the capacity to capture the underlying pattern, so it errs on both train and test sets.
Flash Cards
What is overfitting? — A model that fits training data (including noise) too closely and generalizes poorly to new data.
What is underfitting? — A model too simple to capture the pattern, giving high error on both training and test data.
How do you detect overfitting? — Low training error but a large gap to a high validation/test error.
Name three fixes for overfitting. — More data, regularization (L1/L2 or dropout), and early stopping or a simpler model.