Logistic Regression Cheat Sheet
A cheat sheet for logistic regression covering scikit-learn usage, the sigmoid function, log loss, multiclass classification, and coefficient interpretation.
1 PageBeginnerMar 8, 2026
Fitting with scikit-learn
Train a binary classifier and evaluate it.
python
from sklearn.linear_model import LogisticRegressionfrom sklearn.metrics import classification_report, roc_auc_scoremodel = LogisticRegression(C=1.0, penalty='l2', max_iter=1000)model.fit(X_train, y_train)y_pred = model.predict(X_test)y_proba = model.predict_proba(X_test)[:, 1]print(classification_report(y_test, y_pred))print('AUC:', roc_auc_score(y_test, y_proba))
Sigmoid & Log Loss
The math behind the prediction and objective function.
python
import numpy as npdef sigmoid(z): return 1 / (1 + np.exp(-z))# Binary cross-entropy (log loss)def log_loss(y_true, y_pred): eps = 1e-15 y_pred = np.clip(y_pred, eps, 1 - eps) return -np.mean(y_true * np.log(y_pred) + (1 - y_true) * np.log(1 - y_pred))
Multiclass Classification
Extend logistic regression beyond two classes.
python
model = LogisticRegression(multi_class='multinomial', solver='lbfgs')model.fit(X_train, y_train) # softmax generalizes sigmoid to K classes
Key Concepts
Core theory behind logistic regression.
- Sigmoid function- Maps any real value into (0, 1), interpreted as the probability of the positive class
- Decision boundary- Threshold (default 0.5) on predicted probability used to assign the final class label
- Log loss (cross-entropy)- Convex loss function minimized during training via gradient-based optimization
- Odds ratio- exp(coefficient) gives the multiplicative change in odds per one-unit increase in a feature
- Regularization strength (C)- Inverse of regularization strength in scikit-learn; smaller C means stronger regularization
- Class imbalance- Use class_weight='balanced' or resampling techniques when classes are heavily skewed
Manual Gradient Descent
Derive and implement the gradient update for log loss directly.
python
import numpy as npdef sigmoid(z): return 1 / (1 + np.exp(-z))def fit_logreg(X, y, lr=0.1, n_iters=3000): n, d = X.shape X_b = np.c_[np.ones(n), X] w = np.zeros(d + 1) for i in range(n_iters): p = sigmoid(X_b @ w) grad = (1 / n) * X_b.T @ (p - y) # gradient of binary cross-entropy w -= lr * grad return w
Inference with statsmodels Logit
Get coefficient p-values and confidence intervals for hypothesis testing.
python
import statsmodels.api as smX_sm = sm.add_constant(X_train)logit = sm.Logit(y_train, X_sm).fit()print(logit.summary()) # coef, std err, z, P>|z|print('Odds ratios:', np.exp(logit.params))print('Pseudo R-squared (McFadden):', logit.prsquared)
Threshold Tuning via Precision-Recall
Pick a decision threshold other than 0.5 to match a business cost tradeoff.
python
from sklearn.metrics import precision_recall_curveprecision, recall, thresholds = precision_recall_curve(y_test, y_proba)f1 = 2 * precision * recall / (precision + recall + 1e-12)best_idx = np.argmax(f1[:-1])best_threshold = thresholds[best_idx]y_pred_tuned = (y_proba >= best_threshold).astype(int)print(f'Best threshold: {best_threshold:.3f}, F1: {f1[best_idx]:.3f}')
Probability Calibration
Correct predicted probabilities so they reflect true empirical frequencies.
python
from sklearn.calibration import CalibratedClassifierCV, calibration_curvecalibrated = CalibratedClassifierCV(model, method='isotonic', cv=5)calibrated.fit(X_train, y_train)prob_true, prob_pred = calibration_curve(y_test, y_proba, n_bins=10)# plot prob_pred vs prob_true against the y=x diagonal to visualize miscalibration
Advanced Concepts
Theory beyond basic fit-and-predict for logistic regression.
- Wald test- Tests whether an individual coefficient significantly differs from zero using (coef / std err)^2 against a chi-square distribution
- Likelihood ratio test- Compares nested models via -2*(logL_reduced - logL_full) to test whether added predictors improve fit significantly
- McFadden's pseudo R-squared- 1 - (logL_model / logL_null); unlike OLS R-squared, values of 0.2-0.4 are considered a good fit
- Perfect separation- When a predictor perfectly splits classes, MLE coefficients diverge to infinity; fix with regularization or Firth's penalized likelihood
- Deviance- -2 * log-likelihood; used to compare model fit similarly to residual sum of squares in linear regression
- Calibration curve- Plots predicted probability bins against observed positive rate to check if probabilities are trustworthy, not just ranks
Pro Tip
Don't read raw logistic regression coefficients as changes in probability — exponentiate them to get odds ratios, since each coefficient describes a multiplicative effect on the odds of the positive class, not a direct additive effect on probability.
Was this cheat sheet helpful?
Explore Topics
#LogisticRegression#LogisticRegressionCheatSheet#DataScience#Beginner#FittingWithScikitLearn#SigmoidLogLoss#MulticlassClassification#KeyConcepts#Functions#MachineLearning#CheatSheet#SkillVeris
