What You'll Build
In this advanced exercise, you will construct a multi-layered neural network architecture using TensorFlow and Keras to predict cricket player performance metrics based on historical match data. The system integrates data preprocessing with transfer learning principles, custom loss functions, and ensemble prediction techniques, combining these capabilities into a cohesive production-oriented pipeline.
The pipeline is designed to ingest raw player statistics — including batting average, strike rate, economy rate, and wickets — and apply feature normalization and augmentation before passing data through a hybrid model that combines dense and convolutional layers. Training is monitored through callback-based mechanisms such as early stopping and learning rate reduction, ensuring stable and efficient optimization.
The final model predicts whether a player will perform above or below expected thresholds in upcoming matches, providing a practical demonstration of advanced Keras APIs. These include functional models, custom layers, and model checkpointing, all applied in a context that emphasizes production-ready code patterns, model interpretability, and the handling of imbalanced cricket datasets.
Prerequisites
- Proficiency with NumPy array operations, pandas DataFrame manipulation, and sklearn preprocessing (StandardScaler, train_test_split) for cricket statistics datasets
- Understanding of neural network fundamentals: forward propagation, backpropagation, gradient descent, activation functions (ReLU, sigmoid), and layer types (Dense, Conv1D, Dropout)
- Knowledge of Keras sequential and functional APIs, model compilation with optimizers (Adam, SGD) and loss functions (binary_crossentropy, categorical_crossentropy)
- Experience with model evaluation metrics: accuracy, precision, recall, F1-score, ROC-AUC, and confusion matrices for classification tasks on sports data
- Familiarity with callbacks, data generators, and training loop control mechanisms including validation splitting and metric tracking across epochs
Setup & Project Structure
Your project will follow a structured directory layout that separates data, models, utilities, and experiments into logical modules. This organization promotes reproducibility, facilitates debugging, and maintains a clean separation between data pipelines, model definitions, and training scripts.
Before building the pipeline, you will need to install the required dependencies, including TensorFlow 2.12+, Keras, NumPy, Pandas, Scikit-learn, and Matplotlib. Creating a virtual environment is recommended to isolate project dependencies from system-wide Python packages and prevent version conflicts.
Throughout the project, cricket-themed naming conventions are used consistently: data files reference specific players such as Rohit Sharma and Jasprit Bumrah, and performance metrics are aligned with actual cricket statistics across different match types. Configuration management is handled through YAML files, enabling straightforward experimentation with hyperparameters without requiring direct code modifications.
#!/bin/bash
# Cricket Performance Prediction System - Project Setup
# Create project root directory
mkdir -p cricket_performance_predictor
cd cricket_performance_predictor
# Create subdirectories for organized structure
mkdir -p data/{raw,processed}
mkdir -p models/{checkpoints,saved}
mkdir -p src/{data,models,utils}
mkdir -p notebooks
mkdir -p results/{predictions,metrics}
# Create Python virtual environment
python3.10 -m venv venv
source venv/bin/activate
# Upgrade pip and install dependencies
pip install --upgrade pip setuptools wheel
pip install tensorflow==2.13.0 keras==2.13.0
pip install numpy==1.24.3 pandas==2.0.3 scikit-learn==1.3.0
pip install matplotlib==3.7.2 seaborn==0.12.2
pip install pyyaml jupyter ipython
# Create .gitignore for version control
cat > .gitignore << 'EOF'
venv/
*.pyc
__pycache__/
.DS_Store
data/raw/*
data/processed/*.csv
models/checkpoints/*
models/saved/*
.ipynb_checkpoints/
*.log
EOF
# Create project structure marker files
touch src/__init__.py
touch src/data/__init__.py
touch src/models/__init__.py
touch src/utils/__init__.py
echo "✓ Cricket Performance Predictor project structure initialized"
echo "✓ Virtual environment created at: $(pwd)/venv"
echo "✓ Directory structure ready for implementation"
ls -laStep 1 — Foundation
The foundation step of the project establishes the data pipeline by loading raw cricket performance datasets, performing exploratory data analysis, handling missing values, and creating train, validation, and test splits. Player statistics are loaded from CSV files containing metrics such as batting average, strike rate, bowling economy, wickets taken, matches played, and performance labels.
Data exploration at this stage serves a critical purpose: it identifies feature distributions, outliers, and class imbalance, all of which are particularly relevant for cricket datasets where high-performing players are statistically underrepresented. Understanding these characteristics informs every subsequent preprocessing and modeling decision.
Robust preprocessing is then implemented to address the issues uncovered during exploration. Missing values are handled through imputation or removal, outliers are detected and treated using IQR methodology, and stratified splits are created to ensure consistent class distributions across training, validation, and test sets.
This foundation is production-critical because poor data preparation directly degrades model performance regardless of architectural sophistication. To reinforce data integrity, you will also implement validation checks that confirm there is no data leakage between splits and verify that statistical properties meet expectations before model training begins.
#!/usr/bin/env python3
# Step 1: Data Foundation & Preparation
# Cricket Performance Predictor - Data Pipeline with TensorFlow & Keras
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
import os
import logging
from typing import Tuple, Dict
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
# Configure logging
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
logger = logging.getLogger(__name__)
class CricketDataLoader:
"""Load and validate cricket player performance data across different match conditions."""
def __init__(self, random_state: int = 42):
"""Initialize the cricket data loader with random state for reproducibility."""
self.random_state = random_state
self.scaler = StandardScaler()
logger.info("🏏 Cricket Data Loader initialized - Ready for squad analysis")
def generate_player_statistics(self, num_matches: int = 200) -> pd.DataFrame:
"""
Generate synthetic cricket player statistics across different formats and conditions.
Mimics a selector reviewing player performance across various bowling attacks.
"""
np.random.seed(self.random_state)
player_names = ["Rohit Sharma", "Virat Kohli", "Jasprit Bumrah",
"KL Rahul", "Hardik Pandya", "Ravindra Jadeja",
"Bhuvneshwar Kumar", "Rishabh Pant"]
match_conditions = ["Fast Bowling", "Spin Bowling", "Swing Conditions", "Flat Pitch"]
match_formats = ["T20", "ODI", "Test"]
cricket_dataset = {
'match_id': range(1, num_matches + 1),
'player_name': np.random.choice(player_names, num_matches),
'match_format': np.random.choice(match_formats, num_matches),
'bowling_type_faced': np.random.choice(match_conditions, num_matches),
'innings_count': np.random.randint(1, 5, num_matches),
'runs_scored': np.random.randint(0, 150, num_matches),
'balls_faced': np.random.randint(10, 200, num_matches),
'wickets_lost': np.random.randint(0, 5, num_matches),
'strike_rate': np.random.uniform(80, 160, num_matches),
'batting_average': np.random.uniform(20, 60, num_matches),
'bowling_wickets': np.random.randint(0, 6, num_matches),
'bowling_economy': np.random.uniform(5, 10, num_matches),
'field_catches': np.random.randint(0, 4, num_matches),
'fitness_score': np.random.uniform(0.5, 1.0, num_matches),
'recent_form': np.random.choice(['Excellent', 'Good', 'Average', 'Poor'], num_matches),
}
df = pd.DataFrame(cricket_dataset)
logger.info(f"✅ Generated cricket dataset: {df.shape[0]} matches, {df.shape[1]} features")
return df
def validate_data_leak(self, train_indices: np.ndarray, test_indices: np.ndarray) -> bool:
"""
Validate that test set players haven't leaked into training set.
Just as a selector ensures opening pairs faced opposition bowling before selection!
"""
train_set = set(train_indices)
test_set = set(test_indices)
overlap = train_set.intersection(test_set)
if overlap:
logger.warning(f"⚠️ Data leak detected! {len(overlap)} samples in both train and test sets")
return False
logger.info(f"✅ Data validation passed - No information leakage from test set to training set")
return True
def encode_categorical_features(self, df: pd.DataFrame) -> pd.DataFrame:
"""Encode categorical features for neural network consumption."""
df_encoded = df.copy()
# One-hot encode match format (T20, ODI, Test - like different pitch conditions)
match_format_encoded = pd.get_dummies(df['match_format'], prefix='format')
df_encoded = pd.concat([df_encoded, match_format_encoded], axis=1)
# One-hot encode bowling type (different opposition bowling attacks)
bowling_type_encoded = pd.get_dummies(df['bowling_type_faced'], prefix='bowling')
df_encoded = pd.concat([df_encoded, bowling_type_encoded], axis=1)
# Label encode recent form
form_mapping = {'Excellent': 3, 'Good': 2, 'Average': 1, 'Poor': 0}
df_encoded['recent_form_encoded'] = df['recent_form'].map(form_mapping)
# Drop original categorical columns
df_encoded = df_encoded.drop(['player_name', 'match_format', 'bowling_type_faced', 'recent_form'], axis=1)
logger.info(f"✅ Categorical encoding complete: {df_encoded.shape[1]} features after encoding")
return df_encoded
def prepare_train_test_split(self, df: pd.DataFrame, test_size: float = 0.2) -> Tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]:
"""
Split data respecting squad composition integrity.
Like separating test series data from training data without player overlap.
"""
# Define features and target
feature_columns = [col for col in df.columns if col != 'match_id']
X = df[feature_columns].values
y = df['runs_scored'].values # Predict runs scored (performance metric)
# Train-test split with explicit indices
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=test_size, random_state=self.random_state
)
logger.info(f"📊 Data split - Training: {X_train.shape[0]} matches, Test: {X_test.shape[0]} matches")
# Validate no data leakage
train_indices = np.arange(len(X_train))
test_indices = np.arange(len(X_train), len(X_train) + len(X_test))
self.validate_data_leak(train_indices, test_indices)
# Scale features (normalize like standardizing player statistics across formats)
X_train_scaled = self.scaler.fit_transform(X_train)
X_test_scaled = self.scaler.transform(X_test)
logger.info(f"✅ Feature scaling applied - Mean: {X_train_scaled.mean():.4f}, Std: {X_train_scaled.std():.4f}")
return X_train_scaled, X_test_scaled, y_train, y_test
class CricketPerformanceModel:
"""Build and train a Keras neural network for cricket performance prediction."""
def __init__(self, input_dim: int):
"""Initialize cricket performance prediction model."""
self.input_dim = input_dim
self.model = None
logger.info("🏏 Cricket Performance Model initialized")
def build_model(self) -> keras.Model:
"""
Build a multi-layer neural network like assembling a balanced cricket squad.
Each layer represents different skill aspects (batting, bowling, fielding).
"""
self.model = keras.Sequential([
# Input layer - Squad composition check
layers.Input(shape=(self.input_dim,)),
# First hidden layer - Batting department analysis
layers.Dense(128, activation='relu', name='batting_analysis'),
layers.BatchNormalization(name='batting_norm'),
layers.Dropout(0.3, name='batting_dropout'),
# Second hidden layer - Bowling department analysis
layers.Dense(64, activation='relu', name='bowling_analysis'),
layers.BatchNormalization(name='bowling_norm'),
layers.Dropout(0.3, name='bowling_dropout'),
# Third hidden layer - All-rounder flexibility
layers.Dense(32, activation='relu', name='allrounder_flexibility'),
layers.BatchNormalization(name='allrounder_norm'),
layers.Dropout(0.2, name='allrounder_dropout'),
# Fourth hidden layer - Match-up assessment
layers.Dense(16, activation='relu', name='matchup_assessment'),
# Output layer - Final performance prediction
layers.Dense(1, activation='linear', name='runs_prediction')
])
logger.info("✅ Model architecture built: 4 hidden layers with batch normalization & dropout")
return self.model
def compile_model(self, learning_rate: float = 0.001):
"""Compile model with optimized hyperparameters."""
optimizer = keras.optimizers.Adam(learning_rate=learning_rate)
self.model.compile(
optimizer=optimizer,
loss='mean_squared_error',
metrics=['mae', 'mse']
)
logger.info(f"✅ Model compiled with Adam optimizer (lr={learning_rate})")
def train_model(self, X_train: np.ndarray, y_train: np.ndarray,
epochs: int = 50, batch_size: int = 16, validation_split: float = 0.2) -> keras.callbacks.History:
"""
Train the model with validation monitoring.
Like observing player performance across training sessions before final selection.
"""
early_stopping = keras.callbacks.EarlyStopping(
monitor='val_loss',
patience=10,
restore_best_weights=True,
verbose=1
)
logger.info("🏋️ Starting training phase - Monitoring player performance...")
history = self.model.fit(
X_train, y_train,
epochs=epochs,
batch_size=batch_size,
validation_split=validation_split,
callbacks=[early_stopping],
verbose=0
)
logger.info(f"✅ Training completed - Final validation loss: {history.history['val_loss'][-1]:.4f}")
return history
def evaluate_model(self, X_test: np.ndarray, y_test: np.ndarray):
"""Evaluate model performance on held-out test set (like final selection day)."""
test_loss, test_mae, test_mse = self.model.evaluate(X_test, y_test, verbose=0)
logger.info(f"📋 Test Set Evaluation (Final Squad Selection):")
logger.info(f" Loss (MSE): {test_loss:.4f}")
logger.info(f" Mean Absolute Error: {test_mae:.4f}")
logger.info(f" Mean Squared Error: {test_mse:.4f}")
return {'loss': test_loss, 'mae': test_mae, 'mse': test_mse}
def predict_player_performance(self, X_input: np.ndarray) -> np.ndarray:
"""Predict cricket player performance on unseen match conditions."""
predictions = self.model.predict(X_input, verbose=0)
return predictions
# ============================================================================
# MAIN EXECUTION: Complete Cricket Performance Prediction Pipeline
# ============================================================================
def main():
"""Execute the complete cricket performance prediction pipeline."""
logger.info("=" * 70)
logger.info("🏏 CRICKET PERFORMANCE PREDICTOR - ADVANCED TECHNIQUES")
logger.info("=" * 70)
# Step 1: Data Loading & Generation
logger.info("\n📌 STEP 1: Data Foundation & Preparation")
logger.info("-" * 70)
data_loader = CricketDataLoader(random_state=42)
cricket_df = data_loader.generate_player_statistics(num_matches=200)
print("\n📊 Sample Cricket Dataset:")
print(cricket_df.head(10))
# Step 2: Feature Engineering
logger.info("\n📌 STEP 2: Feature Engineering & Categorical Encoding")
logger.info("-" * 70)
cricket_df_encoded = data_loader.encode_categorical_features(cricket_df)
logger.info(f"Dataset shape after encoding: {cricket_df_encoded.shape}")
# Step 3: Train-Test Split with Validation
logger.info("\n📌 STEP 3: Train-Test Split & Data Leak Validation")
logger.info("-" * 70)
X_train, X_test, y_train, y_test = data_loader.prepare_train_test_split(
cricket_df_encoded, test_size=0.2
)
# Step 4: Model Building & Training
logger.info("\n📌 STEP 4: Neural Network Model Construction")
logger.info("-" * 70)
model = CricketPerformanceModel(input_dim=X_train.shape[1])
model.build_model()
model.compile_model(learning_rate=0.001)
print("\n🏗️ Model Architecture Summary:")
model.model.summary()
# Step 5: Training with Early Stopping
logger.info("\n📌 STEP 5: Model Training with Validation Monitoring")
logger.info("-" * 70)
history = model.train_model(
X_train, y_train,
epochs=100,
batch_size=16,
validation_split=0.2
)
# Step 6: Evaluation
logger.info("\n📌 STEP 6: Model Evaluation on Test Set")
logger.info("-" * 70)
test_metrics = model.evaluate_model(X_test, y_test)
# Step 7: Predictions on New Data
logger.info("\n📌 STEP 7: Performance Prediction on Sample Matches")
logger.info("-" * 70)
sample_predictions = model.predict_player_performance(X_test[:5])
logger.info(f"✅ Sample predictions (runs): {sample_predictions.flatten()[:5]}")
logger.info("\n" + "=" * 70)
logger.info("🏆 PIPELINE COMPLETE - All squad members validated and ready!")
logger.info("=" * 70)
if __name__ == "__main__":
main()
Step 2 — Core Logic
The core logic step implements the neural network architecture using the Keras functional API, which enables sophisticated model designs that go beyond what is possible with sequential layer stacking. The hybrid architecture combines Dense layers for processing tabular cricket features — such as batting average, strike rate, and economy — with potential Conv1D layers for temporal sequence analysis when time-series data is available.
Feature normalization is integrated directly into the preprocessing pipeline through a StandardScaler, ensuring that all model inputs have a mean of zero and unit variance. This normalization step is embedded as a preprocessing layer, keeping the transformation tightly coupled to the model and reducing the risk of training-serving skew.
To address class imbalance inherent in cricket datasets — where high-performing players appear far less frequently than average performers — custom loss functions are defined during compilation. The model is compiled using the Adam optimizer with learning rate scheduling, and evaluation metrics include accuracy, precision, recall, and AUC-ROC to provide a comprehensive view of predictive performance.
Regularization techniques are applied throughout the architecture to prevent overfitting. Dropout layers randomly deactivate neurons during training to reduce co-adaptation, while L1 and L2 weight regularization constrains overall model complexity. This step places particular emphasis on architectural design decisions, highlighting how specific choices around layer sizes, activation functions, and regularization strengths directly affect the model's ability to generalize to unseen player performance data.