What Is Regression in Data Science?
Learn what regression is in data science, how linear and regularized models work, key metrics like R-squared and RMSE, and real interview-ready examples.
Expected Interview Answer
Regression is a supervised learning technique that models the relationship between one or more input variables and a continuous output variable, letting you predict a numeric value rather than a category.
A regression model fits a function — linear, polynomial, or more complex — to observed data by minimizing the error between predicted and actual values, commonly using least squares or gradient-based optimization. Linear regression assumes a straight-line relationship between inputs and the target, while variants like polynomial, ridge, and lasso regression handle curvature or add regularization to control overfitting. Regression underlies forecasting tasks such as predicting sales, house prices, or temperature, and its coefficients often double as interpretable measures of how strongly each feature influences the outcome.
- Predicts continuous numeric outcomes, not just categories
- Coefficients are interpretable and explain feature influence
- Simple to fit, fast to train, and easy to debug
- Extends naturally to regularized and non-linear variants
- Forms the statistical backbone of forecasting and trend analysis
AI Mentor Explanation
Regression is like estimating a batter's final score from overs faced and current run rate — you fit a line through past innings to predict a number, not just win or lose. A coach studying how strike rate rises with balls faced is running a mental regression, and the slope tells him exactly how many extra runs to expect per additional over survived.
How a simple linear regression line is fit to data
Input features (X)
- square footage
- number of bedrooms
- location score
Model fit
- minimize sum of squared errors
- solve for coefficients (weights)
Predicted output (Y)
- continuous numeric value, e.g. house price
Step-by-Step Explanation
Step 1
Define the target
Choose the continuous variable you want to predict, such as price, temperature, or demand.
Step 2
Select features
Pick input variables believed to influence the target and prepare them (scale, encode, clean).
Step 3
Fit the model
Estimate coefficients that minimize the error between predicted and actual values, typically via least squares or gradient descent.
Step 4
Evaluate fit
Check metrics like R-squared, MAE, and RMSE, and inspect residual plots for patterns indicating a poor fit.
Step 5
Regularize if needed
Apply ridge or lasso penalties to control overfitting when features are numerous or correlated.
What Interviewer Expects
- Distinguishes regression (continuous output) from classification (categorical output)
- Can explain least squares and how coefficients are estimated
- Knows common evaluation metrics: R-squared, MAE, RMSE
- Understands assumptions like linearity, independence, and homoscedasticity
- Can mention regularized variants like ridge and lasso
Common Mistakes
- Confusing regression with classification
- Forgetting to check residuals for non-linearity or heteroscedasticity
- Ignoring multicollinearity among features
- Using R-squared alone without considering overfitting
Best Answer (HR Friendly)
“Regression is a data science method for predicting a number, like a price or a temperature, based on other information you already know. It works by finding the best mathematical line or curve that fits historical data, and that pattern is then used to make predictions on new data.”
Code Example
from sklearn.linear_model import LinearRegression
import numpy as np
# Square footage -> price
X = np.array([[800], [1200], [1500], [2000], [2500]])
y = np.array([150000, 210000, 260000, 320000, 400000])
model = LinearRegression()
model.fit(X, y)
print("Slope:", model.coef_[0]) # price increase per sqft
print("Intercept:", model.intercept_)
print("Predicted price for 1800 sqft:", model.predict([[1800]])[0])Follow-up Questions
- What is the difference between linear and logistic regression?
- How do ridge and lasso regression differ from ordinary least squares?
- What does R-squared actually measure?
- How would you detect multicollinearity in your features?
- When would you choose regression over a classification approach?
MCQ Practice
1. What type of output does a regression model predict?
Regression predicts continuous numeric values, unlike classification which predicts discrete categories.
2. Which technique is commonly used to fit a linear regression model?
Ordinary least squares minimizes the sum of squared errors between predicted and actual values to estimate coefficients.
3. What does adding a lasso penalty to regression primarily help with?
Lasso regression adds an L1 penalty that shrinks some coefficients to zero, reducing overfitting and performing feature selection.
Flash Cards
What does regression predict? — A continuous numeric value, such as price or temperature, rather than a discrete category.
How are linear regression coefficients estimated? — By minimizing the sum of squared errors between predicted and actual values (least squares).
What is the difference between ridge and lasso? — Ridge uses an L2 penalty that shrinks coefficients; lasso uses an L1 penalty that can shrink some coefficients to exactly zero.
Name a common regression evaluation metric. — R-squared, mean absolute error (MAE), or root mean squared error (RMSE).