Phase 2 — Classification: Predicting Match Outcomes
Phase 2 flips the prediction target: instead of how many runs the batting team will score, you will predict whether the team chasing will win (binary classification). The franchise team wants a win-probability number between 0 and 1 that updates after the first innings ends — so they can adjust their team talks and bowling selections for the second innings.
Analogy🏏Cricket
🏏 Think of it like cricket: the DLS method gives a target — your model gives a probability. After CSK posts 185, your classifier looks at the final score, the venue, the toss, and the batting depth and answers: 'The chasing team has a 62% chance of winning.' That number shapes the captain's aggressive vs defensive second-innings strategy.
Step 0 — Load Data and Check Class Balance
python
# ── Paste the shared data generator from Lesson 31 here first ────────────────
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split, GridSearchCV, StratifiedKFold
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import (classification_report, roc_auc_score,
confusion_matrix, ConfusionMatrixDisplay,
roc_curve, RocCurveDisplay)
import matplotlib.pyplot as plt
# Features and target for classification
num_feats = ['final_score','powerplay_runs','powerplay_wickets',
'run_rate_10','wickets_at_10','top_order_avg','toss_winner_bats',
'team_batting_rank']
cat_feats = ['venue']
target = 'target_won' # 1 = chasing team wins
X = matches[num_feats + cat_feats]
y = matches[target]
print(f"Class distribution:")
print(y.value_counts(normalize=True).round(3))
print(f"Minority class: {y.mean()*100:.1f}% wins")
Step 1 — Stratified Split and Preprocessing
python
# Stratified split preserves class ratio in train/test
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, stratify=y, random_state=42)
preprocessor = ColumnTransformer([
('num', StandardScaler(), num_feats),
('cat', OneHotEncoder(handle_unknown='ignore'), cat_feats),
])
print(f"Train: {X_train.shape} | Class balance: {y_train.mean():.3f}")
print(f"Test: {X_test.shape} | Class balance: {y_test.mean():.3f}")
Step 2 — Train Multiple Classifiers
python
classifiers = {
'Logistic Regression': Pipeline([
('pre', preprocessor),
('clf', LogisticRegression(max_iter=1000, random_state=42))
]),
'Random Forest': Pipeline([
('pre', preprocessor),
('clf', RandomForestClassifier(n_estimators=200, random_state=42, n_jobs=-1))
]),
'Gradient Boosting': Pipeline([
('pre', preprocessor),
('clf', GradientBoostingClassifier(n_estimators=200, learning_rate=0.05,
max_depth=4, random_state=42))
]),
}
clf_results = {}
skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
print(f"{'Model':<22} {'F1':>6} {'AUC':>6} {'Prec':>6} {'Rec':>6} {'CV-AUC':>8}")
print("-" * 62)
for name, pipe in classifiers.items():
pipe.fit(X_train, y_train)
y_pred = pipe.predict(X_test)
y_proba = pipe.predict_proba(X_test)[:, 1]
from sklearn.metrics import f1_score, precision_score, recall_score
from sklearn.model_selection import cross_val_score
f1 = f1_score(y_test, y_pred)
auc = roc_auc_score(y_test, y_proba)
prec = precision_score(y_test, y_pred)
rec = recall_score(y_test, y_pred)
cv_auc = cross_val_score(pipe, X_train, y_train,
cv=skf, scoring='roc_auc').mean()
clf_results[name] = {'F1': f1, 'AUC': auc, 'CV-AUC': cv_auc}
print(f"{name:<22} {f1:>6.4f} {auc:>6.4f} {prec:>6.4f} {rec:>6.4f} {cv_auc:>8.4f}")
best_clf_name = max(clf_results, key=lambda k: clf_results[k]['AUC'])
print(f"\n✅ Best classifier: {best_clf_name}")
Step 3 — GridSearchCV Hyperparameter Tuning
python
# Tune the best classifier (example assumes RandomForest or GBM wins)
# Adjust param_grid keys based on whichever model won above
param_grid = {
'clf__n_estimators': [100, 200, 300],
'clf__max_depth' : [3, 5, 7, None],
}
grid_search = GridSearchCV(
classifiers[best_clf_name],
param_grid,
cv=skf,
scoring='roc_auc',
n_jobs=-1,
verbose=0
)
grid_search.fit(X_train, y_train)
print(f"Best params : {grid_search.best_params_}")
print(f"Best CV AUC : {grid_search.best_score_:.4f}")
# Evaluate tuned model
best_clf = grid_search.best_estimator_
y_pred_tuned = best_clf.predict(X_test)
y_proba_tuned = best_clf.predict_proba(X_test)[:, 1]
print(f"\nTuned model test AUC: {roc_auc_score(y_test, y_proba_tuned):.4f}")
print("\nClassification Report:")
print(classification_report(y_test, y_pred_tuned, target_names=['Batting team wins','Chasing team wins']))
Step 4 — Confusion Matrix and ROC Curve
python
fig, axes = plt.subplots(1, 2, figsize=(13, 5))
# Confusion matrix
cm = confusion_matrix(y_test, y_pred_tuned)
disp = ConfusionMatrixDisplay(cm, display_labels=['Batting wins','Chasing wins'])
disp.plot(ax=axes[0], colorbar=False, cmap='Blues')
axes[0].set_title(f'Confusion Matrix — {best_clf_name}')
# ROC curve
fpr, tpr, _ = roc_curve(y_test, y_proba_tuned)
auc_val = roc_auc_score(y_test, y_proba_tuned)
axes[1].plot(fpr, tpr, lw=2, label=f'{best_clf_name} (AUC={auc_val:.3f})')
axes[1].plot([0, 1], [0, 1], 'k--', label='Random (AUC=0.500)')
axes[1].set(xlabel='False Positive Rate', ylabel='True Positive Rate',
title='ROC Curve — Match Outcome Classifier')
axes[1].legend(fontsize=9)
plt.tight_layout(); plt.show()
Step 5 — Win Probability Function
python
def predict_win_probability(pipe, final_score, powerplay_runs,
powerplay_wickets, run_rate_10, wickets_at_10,
top_order_avg, toss_winner_bats,
team_batting_rank, venue):
"""Return probability (0–1) that the chasing team wins."""
row = pd.DataFrame([{
'final_score' : final_score,
'powerplay_runs' : powerplay_runs,
'powerplay_wickets' : powerplay_wickets,
'run_rate_10' : run_rate_10,
'wickets_at_10' : wickets_at_10,
'top_order_avg' : top_order_avg,
'toss_winner_bats' : int(toss_winner_bats),
'team_batting_rank' : team_batting_rank,
'venue' : venue,
}])
prob = pipe.predict_proba(row)[0][1]
return round(prob, 4)
# Example: CSK posted 182 at Chepauk
p = predict_win_probability(
best_clf,
final_score=182, powerplay_runs=55, powerplay_wickets=2,
run_rate_10=9.2, wickets_at_10=3, top_order_avg=35,
toss_winner_bats=True, team_batting_rank=3, venue='Chepauk'
)
print(f"Win probability for chasing team: {p*100:.1f}%")
print(f"Batting team win probability : {(1-p)*100:.1f}%")
Phase 2 Deliverables Checklist
Before proceeding to Phase 3, verify: (1) Class balance checked and stratified split used. (2) At least two classifiers compared on F1, ROC-AUC, precision, and recall. (3) GridSearchCV used for at least one hyperparameter on the best classifier. (4) Confusion matrix and ROC curve plotted. (5) `predict_win_probability` function working on at least two test scenarios. If AUC < 0.70, add interaction features (e.g. run_rate_10 × wickets_at_10) or try class_weight='balanced' for imbalanced classes.
Analogy🏏Cricket
🏏 Think of it like cricket: this checklist is the third-umpire review that must clear before you advance to Phase 3. Just as you'd check the match situation is fairly represented before judging a chase — not stacking one side, you confirm class balance is checked and a stratified split used. Just as a fair verdict compares at least two game plans on several honest measures, two or more classifiers must be compared on F1, ROC-AUC, precision and recall. Just as a captain fine-tunes his best bowler's field before the crucial spell, GridSearchCV must tune at least one hyperparameter on the strongest classifier. Just as you'd review both the dismissals tally and the win-probability graph, a confusion matrix and ROC curve must be plotted. Just as you'd rehearse the plan on two match scenarios, predict_win_probability must work on at least two test cases. And just as a weak review — AUC below 0.70 — sends you back to add sharper reads (interaction features), the checklist tells you exactly when the model isn't yet good enough to pass.
- Always use stratify=y in train_test_split for classification to preserve class proportions across splits.
- ROC-AUC is the primary metric for binary classifiers with moderate imbalance — it is threshold-independent and class-ratio-robust.
- GridSearchCV with StratifiedKFold ensures hyperparameter search does not overfit to a lucky split.
- Report precision and recall for each class separately — overall accuracy hides class-specific behaviour.
- The confusion matrix reveals asymmetric errors: false positives (predicted win, actual loss) vs false negatives (predicted loss, actual win) have different business costs.
- Wrap the final classifier in a probability-returning function — win probability (0–1) is more actionable than a binary label.