Core Skills Application Exercise

What You'll Build

In this exercise, you will build an end-to-end deep learning pipeline using TensorFlow and Keras to predict cricket player performance metrics based on historical match statistics. The project integrates data loading, model construction, training with callbacks, and evaluation into a cohesive workflow, culminating in a sequential neural network that ingests features such as runs scored, wickets taken, batting average, and bowling economy, then outputs a performance rating.

This exercise reinforces core TensorFlow concepts including tensor manipulation, layer architecture design, loss function selection, optimization strategies, and metrics tracking. The pipeline encompasses data preprocessing with standardization, model compilation with appropriate loss and optimizer selection, training with early stopping callbacks, and comprehensive evaluation using multiple metrics.

The overall architecture reflects a production-grade workflow that directly mirrors real-world machine learning deployment patterns, where data flows through distinct processing stages before reaching model inference. This structure ensures that the skills practiced here translate directly to professional machine learning engineering contexts.

Analogy🏏Cricket

🏏 Think of it like cricket: Consider a team's coaching staff analyzing a batter's preparation for the upcoming Test series. The coaching team collects detailed statistics: Rohit Sharma's average against fast bowlers in different weather conditions, his boundary percentage in powerplay overs, and his performance trajectory across different venues over five years. The neural network you build is like the team's analytical system: the input layer represents the raw match conditions (opposing team strength, pitch type, weather), the hidden layers work like the coaching strategy—processing and combining information to identify patterns (just as coaches combine multiple data points to predict future form), and the output layer produces the final prediction of expected runs. Training the model with historical data is equivalent to studying past innings footage; the loss function measures how far predictions miss the actual scorecard, and the optimizer adjusts the strategy just as coaches refine tactics based on performance reviews. What this reveals is that both neural networks and cricket teams must learn patterns from history, adapt their approach based on feedback, and balance complex variables to make reliable predictions.

Prerequisites

Understanding of NumPy array operations including reshape, concatenate, and slicing for tensor manipulation
Familiarity with Pandas DataFrames for loading, filtering, and preprocessing structured cricket statistics data
Knowledge of neural network basics: layers, activation functions (ReLU, sigmoid), forward propagation, and backpropagation
Understanding of train-test split methodology and cross-validation concepts for robust model evaluation
Experience with basic TensorFlow/Keras syntax including model instantiation, layer addition, and compile parameters

Setup & Project Structure

Begin by creating a structured project directory with separate folders for data, models, and scripts. This organization pattern mirrors professional machine learning projects where data pipelines, model artifacts, and executable code remain cleanly separated, ensuring that anyone accessing the codebase can understand and navigate it immediately.

Analogy🏏Cricket

🏏 Think of it like cricket: setting up a clean project directory and virtual environment is like a team laying out its dressing room and equipment before a tour begins. Just as a well-run side keeps bats, pads, and match balls in separate labelled kit bags so nobody grabs the wrong gear mid-match, you create isolated folders and a virtual environment so your dependencies — TensorFlow, pandas, numpy, matplotlib, scikit-learn — never clash. Just as a touring team isolates its own supplies rather than borrowing from the host ground, a virtual environment isolates package versions so one project cannot corrupt another. Just as a disorganised dressing room costs precious minutes when a batter is padding up in a hurry, a messy project structure costs hours of debugging later. The payoff: a tidy, isolated setup means every practice session and every model run starts smoothly, the way a well-drilled squad walks onto the field with every piece of kit exactly where it should be.

Install the required dependencies, including TensorFlow with its integrated Keras API, NumPy for numerical operations, Pandas for data manipulation, scikit-learn for preprocessing utilities, and Matplotlib for visualization. Creating a dedicated virtual environment isolates these dependencies and prevents version conflicts with other projects on the same system.

Consistent naming conventions for cricket-themed variables, classes, and files should be applied throughout the project to improve code readability and maintain thematic coherence. This level of discipline also supports reproducibility, ensuring that anyone following the same data loading and processing sequence can regenerate identical results.

bash

#!/bin/bash
# Create project structure for cricket performance prediction ML pipeline

mkdir -p cricket_ml_pipeline/{data,models,notebooks,scripts}
cd cricket_ml_pipeline

# Create virtual environment
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install --upgrade pip
pip install tensorflow==2.13.0 \n            keras==2.13.0 \n            numpy>=1.23.0 \n            pandas>=1.5.0 \n            scikit-learn>=1.2.0 \n            matplotlib>=3.6.0 \n            jupyter>=1.0.0

# Create initial data directory with CSV placeholder
touch data/cricket_player_stats.csv
touch scripts/train_performance_model.py
touch scripts/evaluate_model.py
touch notebooks/exploration.ipynb

echo "Cricket ML Pipeline project structure initialized successfully"
ls -la

Step 1 — Foundation

Step 1 establishes the data foundation by loading cricket player statistics and applying essential preprocessing transformations. You will create a synthetic dataset containing realistic cricket metrics — including runs, wickets, batting average, bowling economy, and strike rate — and implement standardization using scikit-learn's StandardScaler to normalize features to zero mean and unit variance.

This normalization step is critical because neural networks converge faster and more reliably when input features occupy similar numerical scales. Without it, a batting average around 50 would otherwise dominate a bowling economy around 3.5 during gradient descent optimization, introducing systematic bias into the learning process.

Data is then split into training (80%) and testing (20%) sets using stratified sampling where applicable, ensuring that both sets contain representative distributions of performance levels. Finally, data integrity is verified by checking for missing values, confirming shape consistency, and examining statistical summaries, so that clean data flows into model training without introducing numerical instabilities.

Analogy🏏Cricket

🏏 Think of it like cricket: Before any captain takes the field, they must understand their squad's composition—which batsmen performed well in the last series against specific bowling types, which bowlers took wickets in similar conditions, and which all-rounders provide flexibility. This mirrors Foundation Step's exploratory analysis where you examine player statistics across different match conditions and formats. Just as a selector checks that opening pairs have actually faced the opposition's bowling attack (data validation), you verify no leaked information from test sets into training sets. The train/validation/test split mirrors match preparation: training data is like practice matches where players learn conditions, validation data is the warm-up match before the main tournament, and test data is the actual championship match where you can only observe performance (never train on it). When Rohit Sharma's career statistics are processed—removing obvious errors (like negative strike rates), handling missing values from cancelled matches, detecting his outlier performance in the 2019 World Cup—you're preparing signal from noise. Understanding why this foundation matters reveals that no sophisticated ensemble technique can rescue a model built on garbage data: the entire model's trustworthiness rests on properly prepared inputs.

python

import numpy as np
import pandas as pd
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

# STEP 1: DATA FOUNDATION & PREPROCESSING

class CricketDataPipeline:
    """Handles loading, preprocessing, and splitting cricket player statistics."""
    
    def __init__(self, random_state=42):
        self.random_state = random_state
        self.scaler = StandardScaler()
        self.X_train = None
        self.X_test = None
        self.y_train = None
        self.y_test = None
        self.feature_names = None
    
    def create_player_dataset(self):
        """Generate synthetic cricket player performance data."""
        # Cricket player names (famous international players)
        players = [
            'Rohit Sharma', 'Virat Kohli', 'Jasprit Bumrah', 'Pat Cummins',
            'Kane Williamson', 'Steve Smith', 'Babar Azam', 'Ben Stokes',
            'Ravi Ashwin', 'Kagiso Rabada', 'David Warner', 'AB de Villiers',
            'MS Dhoni', 'Joe Root', 'Jason Roy', 'Trent Boult'
        ]
        
        n_players = len(players) * 5  # Multiple seasons per player
        np.random.seed(self.random_state)
        
        cricket_data = {
            'player_name': np.random.choice(players, n_players),
            'innings_count': np.random.randint(10, 150, n_players),
            'runs_scored': np.random.randint(200, 5000, n_players),
            'average': np.random.uniform(20, 60, n_players),
            'strike_rate': np.random.uniform(80, 140, n_players),
            'wickets_taken': np.random.randint(0, 50, n_players),
            'bowling_average': np.random.uniform(15, 50, n_players),
            'economy_rate': np.random.uniform(5, 9, n_players),
            'centuries': np.random.randint(0, 15, n_players),
            'match_id': np.random.randint(1000, 9999, n_players)
        }
        
        df = pd.DataFrame(cricket_data)
        # Create a target: performance rating (0 = underperforming, 1 = excellent performer)
        df['performance_rating'] = (
            (df['average'] > 35) & (df['strike_rate'] > 100) | 
            (df['wickets_taken'] > 15) & (df['bowling_average'] < 30)
        ).astype(int)
        
        return df
    
    def preprocess_data(self, df):
        """
        Preprocess cricket statistics using StandardScaler.
        
        Just like a coach standardizes training intensity (high intensity cardio
        paired with medium intensity skill work) so all players train optimally,
        StandardScaler normalizes diverse cricket metrics to have mean=0 and std=1.
        This ensures runs_scored (0-5000), strike_rate (80-140), and wickets_taken
        (0-50) all contribute equally to model training.
        """
        # Select numerical features for scaling
        feature_columns = [
            'innings_count', 'runs_scored', 'average', 'strike_rate',
            'wickets_taken', 'bowling_average', 'economy_rate', 'centuries'
        ]
        self.feature_names = feature_columns
        
        X = df[feature_columns].values
        y = df['performance_rating'].values
        
        # Split into training and testing sets
        self.X_train, self.X_test, self.y_train, self.y_test = train_test_split(
            X, y, test_size=0.2, random_state=self.random_state
        )
        
        # Fit scaler on training data and transform both sets
        # (Crucial: fit only on training data to prevent data leakage)
        self.X_train = self.scaler.fit_transform(self.X_train)
        self.X_test = self.scaler.transform(self.X_test)
        
        print("✅ Data Preprocessing Complete")
        print(f"Training set shape: {self.X_train.shape}")
        print(f"Test set shape: {self.X_test.shape}")
        print(f"Class distribution (Train): {np.bincount(self.y_train)}")
        print(f"Class distribution (Test): {np.bincount(self.y_test)}")
        
        return self.X_train, self.X_test, self.y_train, self.y_test
    
    def build_player_performance_model(self):
        """Build a Keras neural network for cricket player performance prediction."""
        model = keras.Sequential([
            layers.Dense(64, activation='relu', input_shape=(len(self.feature_names),)),
            layers.BatchNormalization(),
            layers.Dropout(0.3),
            
            layers.Dense(32, activation='relu'),
            layers.BatchNormalization(),
            layers.Dropout(0.2),
            
            layers.Dense(16, activation='relu'),
            
            layers.Dense(1, activation='sigmoid')  # Binary classification
        ])
        
        model.compile(
            optimizer='adam',
            loss='binary_crossentropy',
            metrics=['accuracy', keras.metrics.Precision(), keras.metrics.Recall()]
        )
        
        return model
    
    def train_and_evaluate(self):
        """Train the model and evaluate on test set."""
        model = self.build_player_performance_model()
        
        print("\n🏏 Training Cricket Player Performance Model...")
        history = model.fit(
            self.X_train, self.y_train,
            epochs=50,
            batch_size=8,
            validation_split=0.2,
            verbose=0
        )
        
        print("✅ Training Complete!\n")
        
        # Evaluate on test set
        test_loss, test_accuracy, test_precision, test_recall = model.evaluate(
            self.X_test, self.y_test, verbose=0
        )
        
        print("📊 Test Set Performance:")
        print(f"Accuracy: {test_accuracy:.4f}")
        print(f"Precision: {test_precision:.4f}")
        print(f"Recall: {test_recall:.4f}")
        print(f"Loss: {test_loss:.4f}")
        
        return model, history


# STEP 2: EXECUTION - CORE SKILLS APPLICATION

if __name__ == "__main__":
    # Initialize the pipeline
    pipeline = CricketDataPipeline(random_state=42)
    
    # Step 1: Create cricket player dataset
    print("🏏 Creating Cricket Player Dataset...\n")
    player_df = pipeline.create_player_dataset()
    print(player_df.head(10))
    print(f"\nDataset shape: {player_df.shape}\n")
    
    # Step 2: Preprocess data (scaling and splitting)
    print("📈 Applying StandardScaler (Foundation Building)...\n")
    X_train, X_test, y_train, y_test = pipeline.preprocess_data(player_df)
    
    # Demonstrate scaling effect
    print("\n📊 Scaling Effect Demo:")
    print(f"Before scaling - runs_scored range: {player_df['runs_scored'].min()}-{player_df['runs_scored'].max()}")
    print(f"Before scaling - strike_rate range: {player_df['strike_rate'].min():.2f}-{player_df['strike_rate'].max():.2f}")
    print(f"\nAfter scaling - X_train mean: {X_train.mean(axis=0)[:3]} (should be ~0)")
    print(f"After scaling - X_train std: {X_train.std(axis=0)[:3]} (should be ~1)")
    
    # Step 3: Build and train neural network
    print("\n" + "="*60)
    model, history = pipeline.train_and_evaluate()
    
    print("\n✨ Pipeline Complete! Your cricket player performance model is ready.")

Step 2 — Core Logic

Step 2 constructs the neural network architecture using the Keras Sequential API. The network is designed with an input dimension matching the feature count — six in this cricket dataset — followed by multiple hidden layers with ReLU activation functions for non-linear feature transformation, dropout regularization to prevent overfitting on small-to-medium datasets, and an output layer with linear activation suited to regression tasks.

The model is compiled with mean squared error (MSE) as the loss function, which is appropriate for continuous performance rating prediction, along with the Adam optimizer for adaptive learning rate management and mean absolute error (MAE) as an interpretable tracking metric. An EarlyStopping callback is incorporated during training to monitor validation loss and halt the process when performance plateaus, preventing unnecessary computation and overfitting.

A learning history tracking mechanism records loss and metrics at each epoch, enabling visualization of model convergence patterns over time. Together, these components form the core trainable neural network that learns cricket performance patterns from the preprocessed data.

Analogy🏏Cricket

🏏 Think of it like cricket: Building your neural network architecture is exactly like Rahul Dravid selecting the ideal batting order for different match situations—you're not using the same eleven for T20 powerplay that you'd use for Test match day-4 grind. Your Keras functional model is the batting order itself: opening batsmen (input layers) handle initial aggressive scoring, middle-order batsmen (hidden Dense layers) stabilize and build partnerships, and tail-enders (final layers) specialize in specific situations (all-rounders contribute to both batting and bowling). Just as Hardik Pandya might bowl fewer overs and focus on death-overs hittability when the team has strong bowlers elsewhere, your Dropout layers randomly 'rest' neurons during training to prevent any single pathway from becoming too dominant—this prevents 'overreliance' on one player's form. Your custom loss function mirrors how a coach weighs different performance dimensions: is it more critical to identify elite players (precision in the "1" class) or to avoid incorrectly ranking average performers as below-average (recall)? The learning rate schedule acts like match-day field placements: aggressive initial setup (high learning rate) early when the model needs foundational learning, then gradually defensive positioning (reduced learning rate) later to fine-tune and hold established advantages. Understanding this architectural thinking reveals that model structure isn't mathematical abstraction but strategic composition—each component serves deliberate performance objectives.

python

import numpy as np
import pandas as pd
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers, models, callbacks
import matplotlib.pyplot as plt

# STEP 2: CORE MODEL ARCHITECTURE & TRAINING PIPELINE

class CricketPerformanceModel:
    """Constructs and trains neural network for cricket player performance prediction."""
    
    def __init__(self, input_dim=6, random_state=42):
        self.input_dim = input_dim
        self.random_state = random_state
        self.model = None
        self.history = None
        tf.random.set_seed(random_state)
        np.random.seed(random_state)
        
    def build_model(self):
        """
        Construct sequential neural network with:
        - Input layer (6 features for cricket stats)
        - Hidden layer 1: 64 neurons, ReLU activation
        - Dropout: 0.2 (prevents overfitting)
        - Hidden layer 2: 32 neurons, ReLU activation
        - Dropout: 0.2
        - Hidden layer 3: 16 neurons, ReLU activation
        - Output layer: 1 neuron, linear activation (regression)
        """
        self.model = models.Sequential([
            layers.Dense(64, activation='relu', input_dim=self.input_dim,
                        name='hidden_layer_1'),
            layers.Dropout(0.2, name='dropout_1'),
            
            layers.Dense(32, activation='relu', name='hidden_layer_2'),
            layers.Dropout(0.2, name='dropout_2'),
            
            layers.Dense(16, activation='relu', name='hidden_layer_3'),
            
            layers.Dense(1, activation='linear', name='output_layer')
        ])
        
        print("Model Architecture:")
        self.model.summary()
        
        return self.model
    
    def compile_model(self, learning_rate=0.001):
        """
        Compile model with optimizer, loss function, and metrics.
        """
        optimizer = keras.optimizers.Adam(learning_rate=learning_rate)
        
        self.model.compile(
            optimizer=optimizer,
            loss='mse',  # Mean Squared Error for regression
            metrics=['mae']  # Mean Absolute Error for interpretability
        )
        
        print("\nModel compiled successfully")
        print(f"Optimizer: Adam (lr={learning_rate})")
        print(f"Loss function: Mean Squared Error (MSE)")
        print(f"Metrics: Mean Absolute Error (MAE)")
    
    def train_model(self, X_train, y_train, X_val, y_val,
                   epochs=100, batch_size=32, verbose=1):
        """
        Train model with EarlyStopping callback to prevent overfitting.
        
        EarlyStopping monitors validation loss and halts training when
        performance plateaus for 10 consecutive epochs.
        """
        early_stop = callbacks.EarlyStopping(
            monitor='val_loss',
            patience=10,
            restore_best_weights=True,
            verbose=1
        )
        
        print(f"\nTraining model for up to {epochs} epochs...")
        print(f"Batch size: {batch_size}")
        print(f"EarlyStopping patience: 10 epochs\n")
        
        self.history = self.model.fit(
            X_train, y_train,
            validation_data=(X_val, y_val),
            epochs=epochs,
            batch_size=batch_size,
            callbacks=[early_stop],
            verbose=verbose
        )
        
        print(f"\nTraining completed after {len(self.history.history['loss'])} epochs")
        return self.history
    
    def plot_training_history(self):
        """
        Visualize training and validation loss over epochs.
        """
        if self.history is None:
            print("No training history available. Train model first.")
            return
        
        fig, axes = plt.subplots(1, 2, figsize=(14, 5))
        
        # Loss plot
        axes[0].plot(self.history.history['loss'], label='Training Loss', linewidth=2)
        axes[0].plot(self.history.history['val_loss'], label='Validation Loss', linewidth=2)
        axes[0].set_xlabel('Epoch', fontsize=12)
        axes[0].set_ylabel('MSE Loss', fontsize=12)
        axes[0].set_title('Model Loss Over Time', fontsize=14, fontweight='bold')
        axes[0].legend(fontsize=10)
        axes[0].grid(True, alpha=0.3)
        
        # MAE plot
        axes[1].plot(self.history.history['mae'], label='Training MAE', linewidth=2)
        axes[1].plot(self.history.history['val_mae'], label='Validation MAE', linewidth=2)
        axes[1].set_xlabel('Epoch', fontsize=12)
        axes[1].set_ylabel('Mean Absolute Error', fontsize=12)
        axes[1].set_title('Model MAE Over Time', fontsize=14, fontweight='bold')
        axes[1].legend(fontsize=10)
        axes[1].grid(True, alpha=0.3)
        
        plt.tight_layout()
        plt.savefig('training_history.png', dpi=150, bbox_inches='tight')
        print("\nTraining history plot saved as 'training_history.png'")
        plt.show()


# EXECUTION WITH SAMPLE DATA
if __name__ == "__main__":
    print("="*70)
    print("STEP 2: CORE MODEL ARCHITECTURE & TRAINING")
    print("="*70)
    
    # Create sample data (using data from Step 1)
    np.random.seed(42)
    n_samples = 500
    X = np.random.randn(n_samples, 6).astype(np.float32)
    y = (2*X[:, 0] + 1.5*X[:, 1] - 0.5*X[:, 2] + 
         1.2*X[:, 3] + 0.8*X[:, 4] - 0.3*X[:, 5] + 
         np.random.randn(n_samples)*0.1).astype(np.float32)
    y = np.clip(y, 0, 100)  # Clip to realistic performance range
    
    # Split into train and validation
    X_train, X_val, y_train, y_val = train_test_split(
        X, y, test_size=0.2, random_state=42
    )
    
    # Initialize and build model
    print("\nInitializing Cricket Performance Prediction Model...")
    model_trainer = CricketPerformanceModel(input_dim=6, random_state=42)
    
    print("\n" + "="*70)
    print("Building neural network architecture...")
    model_trainer.build_model()
    
    # Compile
    print("\n" + "="*70)
    print("Compiling model...")
    model_trainer.compile_model(learning_rate=0.001)
    
    # Train
    print("\n" + "="*70)
    print("Training model...")
    history = model_trainer.train_model(
        X_train, y_train,
        X_val, y_val,
        epochs=100,
        batch_size=32,
        verbose=0  # Set to 1 for detailed output
    )
    
    # Plot training history
    print("\n" + "="*70)
    print("Plotting training metrics...")
    model_trainer.plot_training_history()
    
    print("\n✓ Step 2 Complete: Model trained successfully")

Step 3 — Integration & Enhancement

Step 3 integrates all previous components into a unified, production-ready evaluation pipeline while adding predictive capabilities and model persistence. Comprehensive testing is performed on the held-out test set, calculating multiple regression metrics — Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and R-squared (R²) coefficient — to assess model generalization across different evaluative perspectives.

Individual predictions are generated for test samples, enabling analysis of model confidence and error patterns across different performance rating ranges. Visualization comparing predicted versus actual performance ratings is created to highlight where the model succeeds and where systematic biases emerge.

The pipeline also includes model serialization using Keras' native save() method and the HDF5 format, allowing trained weights and architecture to be persisted for production deployment without retraining. Advanced enhancements such as custom prediction functions for new player data, uncertainty quantification, and batch processing capabilities further transform the trained model from an experimental artifact into a deployable system suitable for real-world cricket analytics applications.

Analogy🏏Cricket

🏏 Think of it like cricket: After a cricket team's internal training (Step 2), they must validate their preparation through actual match performance (Step 3). During the IPL season, a team doesn't just practice in nets; they play matches, accumulate statistics, analyze scorecard data, and assess whether their strategies actually work against real opposition—this is the integration phase. When Mumbai Indians review their batting performance across the entire tournament, they calculate batting averages, strike rates, and average runs per innings for each player—parallel to calculating MAE and RMSE to understand prediction error patterns. The model persistence (saving weights) mirrors how a team documents successful strategies for future seasons: if Rohit Sharma discovers an effective technique against left-arm spinners during the tournament, the team archives this knowledge (strategy documentation) and reapplies it in future matches without rediscovering it from scratch. Visualization of predictions versus actuals functions like match replays—watching the difference between planned field placements and actual batsman positioning reveals whether the strategy was sound but poorly executed, or fundamentally flawed. The batch processing capability mirrors how franchises use historical match data to profile opposing teams in real-time rather than analyzing each match in isolation. By integrating these elements, you transform a trained neural network into a cricket analytics tool that stakeholders can trust and deploy for actual player selection and strategy decisions.