Precision, Recall, and F1 Score

Analogy categoryAnalogies

Precision and recall are the two most important metrics for evaluating a classifier on the class that actually matters, rather than on aggregate correctness. Precision answers: 'Of everything I labeled positive, how much was actually positive?' It is computed as TP / (TP + FP). Recall (also called sensitivity or true positive rate) answers a different question: 'Of everything that was actually positive, how much did I find?' It is computed as TP / (TP + FN). These two metrics are usually in tension — a classifier tuned to catch every possible positive case (high recall) will typically also flag more false alarms (lower precision), and vice versa.

🏏

Cricket analogy: A wicketkeeper's precision is how many of his stumping appeals were actually out, while recall is how many actual dismissals he correctly appealed for; a keeper who appeals every close chance boosts recall but tanks precision with too many wrong shouts.

The Precision-Recall Tradeoff

Most classifiers output a probability or score, which is converted to a class label by applying a threshold (commonly 0.5). Lowering the threshold means more examples get labeled positive, which increases recall (fewer positives are missed) but typically decreases precision (more false alarms slip through). Raising the threshold does the opposite. This tradeoff is why a single precision or recall number, in isolation, is rarely sufficient — practitioners plot precision against recall across all thresholds (a precision-recall curve) to understand the full tradeoff and pick a threshold appropriate to the application's cost structure.

🏏

Cricket analogy: Lowering a DRS review threshold means umpires review more decisions, catching more actual wrong calls (higher recall) but wasting more reviews on marginal calls (lower precision), which is why teams weigh review success rate across confidence levels before calling for it.

F1 Score: A Harmonic Balance

The F1 score combines precision and recall into a single number using the harmonic mean: F1 = 2 * (precision * recall) / (precision + recall). The harmonic mean is used instead of the arithmetic mean because it punishes extreme imbalance between the two — a classifier with precision 1.0 and recall 0.01 would score an arithmetic mean of about 0.5, which overstates its usefulness, while its F1 score would be near 0.02, correctly reflecting that it is nearly useless. F1 is most appropriate when false positives and false negatives carry roughly equal cost; when they don't, a weighted variant (F-beta) or examining precision and recall separately is more appropriate.

🏏

Cricket analogy: A bowler with a wicket-taking precision of 1.0 but recall of 0.01 has bowled one perfect delivery all series; F1's harmonic mean correctly rates him as nearly useless, unlike an arithmetic average that would flatter his lone success.

python

from sklearn.metrics import precision_score, recall_score, f1_score, fbeta_score, precision_recall_curve
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.datasets import make_classification

X, y = make_classification(n_samples=1000, weights=[0.85, 0.15], random_state=1)
X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, random_state=1)

model = LogisticRegression().fit(X_train, y_train)
probs = model.predict_proba(X_test)[:, 1]

# Standard 0.5 threshold
y_pred = (probs >= 0.5).astype(int)
print('precision:', precision_score(y_test, y_pred))
print('recall:', recall_score(y_test, y_pred))
print('f1:', f1_score(y_test, y_pred))

# F2 score weights recall twice as heavily as precision
print('f2 (recall-weighted):', fbeta_score(y_test, y_pred, beta=2))

precisions, recalls, thresholds = precision_recall_curve(y_test, probs)

In medical screening, recall is often prioritized over precision — missing a sick patient (false negative) is usually far more costly than a false alarm that leads to a follow-up test (false positive). In spam filtering, the opposite is often true: incorrectly binning an important email as spam (false positive) can be worse than letting one spam message through (false negative).

A common mistake is reporting F1 score for a multi-class problem without specifying the averaging method ('macro', 'micro', or 'weighted'). Each gives a materially different number on imbalanced multi-class data, so always state which averaging scheme was used.

Precision = TP / (TP + FP): how trustworthy are the positive predictions?
Recall = TP / (TP + FN): how many actual positives did the model find?
Precision and recall usually trade off against each other as the classification threshold changes.
F1 is the harmonic mean of precision and recall, penalizing extreme imbalance between the two.
F-beta generalizes F1 to weight recall or precision more heavily depending on application costs.
The right metric to optimize depends on the relative cost of false positives versus false negatives in the specific problem.

Practice what you learned

1. What does precision measure?

2. Lowering the classification threshold typically has what effect?

3. Why does F1 score use the harmonic mean rather than the arithmetic mean of precision and recall?

4. In which scenario would you most likely prioritize recall over precision?

5. What does the F-beta score with beta > 1 emphasize?

Was this page helpful?

Confusion Matrix and Classification Metrics

ROC Curves and AUC

Topics covered

#Python#MachineLearningBasicsStudyNotes#MachineLearning#PrecisionRecallAndF1Score#Precision#Recall#Score#Tradeoff#StudyNotes#SkillVeris

Precision, Recall, and F1 Score