How Does Linear Regression Work?
Learn how linear regression works: the line equation, mean squared error, gradient descent, key assumptions, and a scikit-learn example for interviews.
Expected Interview Answer
Linear regression models the relationship between input features and a continuous target by fitting a straight-line equation, y = w1x1 + w2x2 + ... + b, that best predicts the target from the inputs.
It learns the weights (coefficients) and bias (intercept) that minimize a loss function, usually mean squared error, the average squared gap between predictions and actual values. This is solved either analytically with the normal equation or iteratively with gradient descent. Each coefficient shows how much the prediction changes per unit change in that feature, which makes the model highly interpretable. It assumes a roughly linear relationship, independent errors, and constant error variance.
- Simple, fast to train, and easy to interpret
- Coefficients quantify each feature's effect
- Strong baseline before trying complex models
- Works well when relationships are roughly linear
- Requires little tuning and few resources
AI Mentor Explanation
A coach predicts a batter's runs from balls faced by fitting the best trend line through past innings. The line's slope says how many runs per ball to expect, exactly as linear regression fits a line whose slope quantifies how the target moves with each input.
Step-by-Step Explanation
Step 1
Define the hypothesis
Assume y is a linear combination of features plus a bias: y = w1x1 + ... + wnxn + b.
Step 2
Choose a loss
Use mean squared error, the average squared difference between predictions and true values.
Step 3
Fit the parameters
Solve for weights via the normal equation or iteratively with gradient descent.
Step 4
Evaluate
Measure fit with R-squared, RMSE, or MAE on held-out data.
Step 5
Interpret and predict
Read each coefficient as a feature's effect, then predict on new inputs.
What Interviewer Expects
- The line equation with weights and bias
- Mean squared error as the loss function
- Awareness of normal equation vs gradient descent
- Key assumptions like linearity and independent errors
- How to interpret coefficients
Common Mistakes
- Confusing linear regression with logistic regression
- Ignoring assumptions like linearity and homoscedasticity
- Forgetting the intercept (bias) term
- Interpreting correlation as causation from coefficients
- Not scaling features before gradient descent
Best Answer (HR Friendly)
“Linear regression predicts a number by drawing the best straight line through your data. The line's slope tells you how much the result changes as an input changes, making it a simple, transparent way to forecast values like prices or sales.”
Code Example
import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.metrics import r2_score
# Feature: hours studied, Target: exam score
X = np.array([[1], [2], [3], [4], [5]])
y = np.array([52, 60, 71, 79, 88])
model = LinearRegression()
model.fit(X, y)
print('slope (weight):', model.coef_[0])
print('intercept (bias):', model.intercept_)
preds = model.predict(X)
print('R-squared:', r2_score(y, preds))
print('predict 6 hours:', model.predict([[6]])[0])Follow-up Questions
- What is the difference between linear and logistic regression?
- How does gradient descent minimize the loss function?
- What does R-squared tell you about the fit?
- What are the key assumptions of linear regression?
- How do you handle multicollinearity between features?
MCQ Practice
1. Linear regression predicts what kind of target?
Linear regression outputs a continuous value; classification tasks use models like logistic regression instead.
2. Which loss function does ordinary linear regression minimize?
Ordinary least squares minimizes the mean squared error between predictions and actual values.
3. What does a coefficient (weight) represent?
Each coefficient measures how much the predicted target changes when that feature increases by one unit, holding others fixed.
Flash Cards
What does linear regression predict? — A continuous numeric target modeled as a weighted linear combination of features plus a bias term.
What loss does it minimize? — Mean squared error, the average squared difference between predicted and actual values.
Normal equation vs gradient descent? — The normal equation solves for weights in closed form; gradient descent finds them iteratively, better for large datasets.
What does a coefficient mean? — The expected change in the target per one-unit increase in that feature, with other features held constant.