Evaluation Metrics Overview — Machine Learning with Scikit-learn

Choosing the right evaluation metric is as important as choosing the right model, because a metric that does not align with the business objective will point the analyst toward the wrong model even when many metrics are computed. Accuracy is the default metric people reach for, but it is often misleading: on a dataset where ninety-five percent of cases are negative, a model that always predicts negative achieves ninety-five percent accuracy while being completely useless. Understanding the full landscape of metrics — accuracy, precision, recall, F1, AUC-ROC, MAE, RMSE, and R2 — their properties, and when each is appropriate is what allows a data scientist to align model optimisation with the actual business goal and to recognise when a model that looks good on a standard metric will fail in deployment because the metric does not capture what matters.

Analogy🏏Cricket

🏏 Think of it like cricket: A batting coach who teaches only one shot — the forward defensive — gives advice with high bias: it is consistently wrong for deliveries that demand a drive or a pull, regardless of how much practice the batsman does. A coach who memorises every ball of the batsman's training career gives advice with high variance: he predicts each training ball perfectly but fails completely on new balls from a different bowler, because he learned the noise of that specific bowler rather than the underlying principles. The great coach finds the balance — teaching the core principles that generalise, without over-specifying for the particular training environment. Just as great coaching lies between the extremes, great ML models balance bias and variance.