What is a Random Forest?
Learn how random forests combine bagged decision trees to cut variance and boost accuracy, with a scikit-learn example and interview tips.
Expected Interview Answer
A random forest is an ensemble model that trains many decision trees on random subsets of the data and features, then combines their predictions by majority vote (classification) or averaging (regression) to produce a more accurate, stable result.
It uses bagging (bootstrap aggregating): each tree sees a random bootstrap sample of the rows, and at every split only a random subset of features is considered. This decorrelates the trees so their errors cancel out when aggregated, dramatically reducing the high variance of a single deep tree while keeping bias low. Random forests also provide feature-importance estimates and an out-of-bag error estimate for free, and they rarely overfit as more trees are added.
- Much higher accuracy than a single tree
- Reduces variance and resists overfitting
- Provides feature-importance rankings
- Handles non-linear data with minimal tuning
- Gives a free out-of-bag validation estimate
AI Mentor Explanation
Instead of trusting one selector, a board polls many analysts, each shown a different slice of matches and stats, then picks the squad by majority vote. A random forest does this with decision trees: each tree sees a random sample of data and features, and their combined vote is far steadier than any single opinion.
Step-by-Step Explanation
Step 1
Bootstrap the data
Draw many random samples with replacement, one dataset per tree.
Step 2
Grow decorrelated trees
Train a decision tree on each sample, considering only a random subset of features at each split.
Step 3
Let trees grow deep
Each tree is grown with little pruning so it has low bias but high variance individually.
Step 4
Aggregate predictions
Combine trees by majority vote for classification or averaging for regression.
Step 5
Estimate error and importance
Use out-of-bag samples for validation and measure feature importance across the ensemble.
What Interviewer Expects
- Understands bagging and bootstrap sampling
- Explains why random feature subsets decorrelate trees
- Knows aggregation reduces variance without raising bias much
- Mentions out-of-bag error and feature importance
- Contrasts random forests with boosting
Common Mistakes
- Confusing random forests (bagging) with boosting methods like gradient boosting
- Forgetting the random feature selection at each split, not just row sampling
- Claiming more trees cause overfitting
- Not knowing how out-of-bag error works
- Assuming individual trees are pruned shallow when they are usually grown deep
Best Answer (HR Friendly)
“A random forest combines the answers of many decision trees, each trained on slightly different data, and takes a vote. Because the trees make different mistakes, averaging them gives a more accurate and reliable prediction than any single tree.”
Code Example
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
X, y = load_wine(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42
)
model = RandomForestClassifier(
n_estimators=200, max_features='sqrt',
oob_score=True, random_state=42
)
model.fit(X_train, y_train)
preds = model.predict(X_test)
print('Test accuracy:', accuracy_score(y_test, preds))
print('OOB score:', model.oob_score_)
print('Top feature importances:', model.feature_importances_[:5])Follow-up Questions
- What is the difference between bagging and boosting?
- How does out-of-bag error estimate generalization?
- Why does random feature selection improve the ensemble?
- How do random forests measure feature importance?
- When would you choose gradient boosting over a random forest?
MCQ Practice
1. Random forests reduce error primarily by lowering:
Averaging many decorrelated deep trees reduces variance while keeping bias low, improving generalization.
2. What technique gives each tree its training data in a random forest?
Bagging draws bootstrap samples (random rows with replacement) so each tree trains on a different dataset.
3. At each split, a random forest tree considers:
Limiting each split to a random feature subset decorrelates the trees so their errors cancel when aggregated.
Flash Cards
What ensemble technique does a random forest use? — Bagging (bootstrap aggregating) over many decision trees.
Why sample features at each split? — It decorrelates the trees so their errors cancel, lowering variance.
What is out-of-bag error? — Validation using samples each tree did not train on, a free generalization estimate.
Does adding more trees cause overfitting? — No — more trees stabilize predictions and generally do not overfit.