Logistic Regression Explained Simply

1What Is Logistic Regression?

Logistic regression is a classification method that predicts the probability that an observation belongs to a particular class, such as spam versus not spam. It takes a linear combination of your inputs and squashes it through the sigmoid function, producing a value between 0 and 1 that reads as a probability.

Despite the word regression in its name, it is used for classification. You set a threshold usually 0.5 and label anything above it as the positive class. It is one of the most reliable and widely deployed models in data science.

2The Sigmoid Function

The heart of logistic regression is the sigmoid, an S-shaped curve that maps any real number to a value between 0 and 1. Large positive inputs approach 1, large negative inputs approach 0, and an input of 0 maps to exactly 0.5.

This is what lets the model output valid probabilities. A linear model on its own could predict values like 1.7 or negative numbers, which make no sense as probabilities. The sigmoid gently bends those outputs into the 0-to-1 range.

3How It Learns

Instead of minimizing squared error like linear regression, logistic regression maximizes the likelihood of the observed labels. In practice it minimizes a loss called log loss or cross-entropy, which heavily penalizes confident wrong predictions.

An optimizer such as gradient descent adjusts the coefficients until the predicted probabilities line up as closely as possible with the actual 0 and 1 labels in the training data. Libraries handle this automatically, so you rarely touch the math directly.

4Fitting a Model in Python

Scikit-learn keeps the workflow nearly identical to linear regression create the model, fit it, and predict. The key difference is that you can ask for probabilities as well as labels.

from sklearn.linear_model import LogisticRegression
model = LogisticRegression()
model.fit(X_train, y_train)
labels = model.predict(X_test)          # 0 or 1
probs = model.predict_proba(X_test)[:, 1]  # probability of class 1

5Interpreting the Coefficients

Logistic regression coefficients describe how each feature affects the odds of the positive outcome. A positive coefficient increases the odds; a negative one decreases them. Exponentiating a coefficient gives an odds ratio, which is easy to explain to non-technical stakeholders.

• Positive coefficient: as the feature rises, the outcome becomes more likely.
• Negative coefficient: as the feature rises, the outcome becomes less likely.
• Odds ratio (e^coefficient): the multiplicative change in odds per unit of the feature.
• Coefficient near zero: that feature has little influence on the prediction.

6Thresholds and Metrics

The default 0.5 threshold is not always right. If missing a positive case is costly such as a disease screen you might lower the threshold to catch more positives, accepting more false alarms.

• Precision: of the cases predicted positive, how many truly are.
• Recall: of the true positives, how many the model caught.
• ROC-AUC: overall ability to separate classes across all thresholds.
• Confusion matrix: the raw counts of correct and incorrect predictions per class.

7Common Mistakes to Avoid

A few missteps commonly trip people up with logistic regression.

• Judging the model by accuracy alone on imbalanced datasets.
• Forgetting to scale features when using regularization, which is on by default in scikit-learn.
• Interpreting coefficients as probabilities they relate to log odds, not raw probability.
• Leaving the threshold at 0.5 without considering the cost of each error type.
• Using it on clearly non-linear boundaries without adding interaction or polynomial features.

8Key Takeaways

The essentials of logistic regression come down to these points.

• It predicts the probability of a binary outcome using the sigmoid function.
• Output is between 0 and 1; a threshold converts it to a class label.
• It is a classification method despite the regression in its name.
• Coefficients relate to odds, giving interpretable odds ratios.
• On imbalanced data, judge it with precision, recall, and AUC, not accuracy.

9Frequently Asked Questions

Q: Is logistic regression a classification or regression algorithm? A: It is a classification algorithm. The name comes from its mathematical roots, but it predicts the probability of belonging to a class, which you then turn into a category label. It does not predict continuous values the way linear regression does.

Q: Can logistic regression handle more than two classes? A: Yes. Multinomial logistic regression, sometimes called softmax regression, extends the idea to several classes at once. Scikit-learn handles this automatically when your target has more than two categories.

Q: Why not just use linear regression for classification? A: Linear regression can produce predictions below 0 or above 1, which are invalid as probabilities, and it handles class boundaries poorly. The sigmoid in logistic regression keeps outputs in a valid probability range and models the decision boundary far better.

Q: When should I change the classification threshold? A: Change it when the costs of false positives and false negatives differ. For medical screening you might lower the threshold to catch more true cases, while for a spam filter you might raise it to avoid blocking legitimate email.