Classification vs Regression in ML
Classification vs regression for ML interviews: discrete labels vs continuous values, loss functions, metrics, and a scikit-learn code example.
Expected Interview Answer
Classification predicts a discrete category or class label, while regression predicts a continuous numeric value. Both are supervised learning tasks; the difference is the type of target being predicted.
In classification the output space is a finite set of classes, such as spam vs not-spam or one of several digit labels, and models output probabilities that are thresholded into a class. In regression the output is a real number on a continuous scale, such as price or temperature. They differ in loss functions (cross-entropy vs mean squared error) and in evaluation metrics (accuracy, precision, recall vs RMSE, MAE, R-squared).
- Classification cleanly handles category decisions like yes/no or multi-class
- Regression produces precise numeric estimates
- Each has well-established, interpretable evaluation metrics
- Correct framing picks the right loss and model
- Many algorithms support both with a suitable output layer
AI Mentor Explanation
Classification is an umpire deciding out or not-out, a discrete verdict from a fixed set of outcomes. Regression is a commentator predicting the exact final score, a continuous number on a sliding scale. Both use the same match evidence, but one outputs a category and the other outputs a precise value, which is exactly how classification and regression differ.
Step-by-Step Explanation
Step 1
Identify the target type
Ask whether the output is a category (discrete) or a number on a continuous scale.
Step 2
Choose the task
Discrete target means classification; continuous target means regression.
Step 3
Select the loss
Classification typically uses cross-entropy; regression typically uses mean squared error or mean absolute error.
Step 4
Pick a model
Many algorithms do both: logistic regression classifies, linear regression predicts numbers, and trees/forests handle either.
Step 5
Evaluate with the right metrics
Use accuracy, precision, recall, F1 for classification; RMSE, MAE, R-squared for regression.
What Interviewer Expects
- Discrete-label vs continuous-value distinction stated clearly
- Correct example tasks for each
- Awareness of differing loss functions
- Correct evaluation metrics for each type
- Knowing both are supervised learning
Common Mistakes
- Using accuracy to evaluate a regression model
- Confusing logistic regression with linear regression by name
- Treating an ordinal or count target carelessly
- Applying RMSE to a pure classification task
- Assuming a model can only do one of the two tasks
Best Answer (HR Friendly)
“Classification is when the computer sorts things into groups, like deciding if an email is spam or not. Regression is when it predicts an actual number, like the price of a house. Same idea of learning from data, but one gives a category and the other gives a number.”
Code Example
from sklearn.datasets import load_breast_cancer, load_diabetes
from sklearn.linear_model import LogisticRegression, LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, mean_squared_error
# Classification: discrete class target
Xc, yc = load_breast_cancer(return_X_y=True)
Xc_tr, Xc_te, yc_tr, yc_te = train_test_split(Xc, yc, random_state=0)
clf = LogisticRegression(max_iter=5000).fit(Xc_tr, yc_tr)
print('Accuracy:', accuracy_score(yc_te, clf.predict(Xc_te)))
# Regression: continuous numeric target
Xr, yr = load_diabetes(return_X_y=True)
Xr_tr, Xr_te, yr_tr, yr_te = train_test_split(Xr, yr, random_state=0)
reg = LinearRegression().fit(Xr_tr, yr_tr)
print('RMSE:', mean_squared_error(yr_te, reg.predict(Xr_te)) ** 0.5)Follow-up Questions
- Why can't you use accuracy to evaluate a regression model?
- How does logistic regression perform classification despite its name?
- What is the difference between MAE and RMSE?
- How do you handle a multi-class classification problem?
- Can a regression output be converted into a classification decision?
MCQ Practice
1. Which problem is a regression task?
Predicting temperature is a continuous numeric output, which makes it a regression task.
2. Which metric is appropriate for classification?
F1 score balances precision and recall, a standard metric for classification tasks.
3. Despite its name, logistic regression is used for:
Logistic regression outputs class probabilities and is a classification algorithm.
Flash Cards
Classification vs regression in one line? — Classification predicts a discrete category; regression predicts a continuous number.
Name two classification metrics. — Accuracy and F1 score (also precision, recall).
Name two regression metrics. — RMSE and MAE (also R-squared).
Is logistic regression classification or regression? — Classification — it outputs class probabilities despite its name.