How Does Logistic Regression Work?
Learn how logistic regression predicts binary outcomes using the sigmoid, log-loss, and gradient descent, with a clear scikit-learn example and interview tips.
Expected Interview Answer
Logistic regression is a supervised classification algorithm that models the probability of a binary outcome by passing a linear combination of the inputs through the sigmoid function, producing a value between 0 and 1.
It computes z = w·x + b, then applies the sigmoid 1/(1+e^-z) to squash z into a probability. A threshold (typically 0.5) converts that probability into a class label. The weights are learned by minimizing the log-loss (binary cross-entropy) using gradient descent, which penalizes confident wrong predictions heavily. Despite its name, it is a classifier, not a regression model, and the decision boundary it learns is linear in the feature space.
- Outputs calibrated probabilities, not just labels
- Fast to train and cheap to predict
- Highly interpretable coefficients (log-odds)
- Strong baseline for binary classification
- Extends to multi-class via softmax
AI Mentor Explanation
A selector rates a batter's chance of scoring a fifty by weighing form, pitch, and opposition, then squashing that score into a 0-to-100% confidence. Logistic regression does the same: it sums weighted signals and bends the total through a sigmoid so the final number is always a sensible probability of the 'out' or 'not out' class.
Step-by-Step Explanation
Step 1
Compute the linear score
Calculate z = w·x + b, a weighted sum of the input features plus a bias term.
Step 2
Apply the sigmoid
Pass z through σ(z) = 1/(1+e^-z) to map it into a probability between 0 and 1.
Step 3
Define the loss
Measure error with log-loss (binary cross-entropy), which punishes confident wrong predictions.
Step 4
Optimize weights
Use gradient descent to iteratively adjust w and b to minimize the loss.
Step 5
Threshold to classify
Convert the probability into a class label using a cutoff, commonly 0.5.
What Interviewer Expects
- Knows it is a classifier, not regression despite the name
- Can explain the sigmoid and its 0-1 output
- Understands log-loss instead of MSE
- Mentions the linear decision boundary
- Can relate coefficients to log-odds
Common Mistakes
- Calling it a regression algorithm for continuous targets
- Using mean squared error instead of log-loss
- Forgetting to scale features before training
- Assuming it can model non-linear boundaries without feature engineering
- Ignoring class imbalance when choosing the threshold
Best Answer (HR Friendly)
“Logistic regression is a simple, popular model that predicts the probability of a yes/no outcome, like whether an email is spam. It weighs the input factors, squeezes the result into a 0-to-100% chance, and then picks a class based on that probability.”
Code Example
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import accuracy_score
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42
)
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
model = LogisticRegression(max_iter=1000)
model.fit(X_train, y_train)
probs = model.predict_proba(X_test)[:, 1]
preds = model.predict(X_test)
print('Accuracy:', accuracy_score(y_test, preds))Follow-up Questions
- Why do we use log-loss instead of mean squared error?
- How does logistic regression extend to multi-class problems?
- What is the interpretation of a coefficient in terms of odds?
- How does L1 vs L2 regularization affect the model?
- How would you handle class imbalance?
MCQ Practice
1. What function does logistic regression use to map scores to probabilities?
The sigmoid function squashes any real number into the range 0 to 1, giving a valid probability.
2. Which loss function is used to train logistic regression?
Log-loss penalizes confident but wrong probability estimates and is the standard objective for logistic regression.
3. The decision boundary learned by standard logistic regression is:
Because the score is a linear combination of features, the boundary where probability equals 0.5 is linear.
Flash Cards
Is logistic regression a classifier or a regressor? — A classifier — it predicts class probabilities for categorical outcomes despite its name.
What is the sigmoid function? — σ(z) = 1/(1+e^-z), which maps any real value into the range 0 to 1.
What loss does it minimize? — Log-loss (binary cross-entropy), optimized via gradient descent.
What do the coefficients represent? — The change in the log-odds of the positive class per unit change in a feature.