Practice — image classifier with transfer learning

What You'll Build

You will build a complete cricket shot image classifier using transfer learning with EfficientNetB0 pretrained on ImageNet. The classifier identifies five IPL batting shot types from match footage frames: cover drive, pull shot, sweep, flick, and straight drive. The project covers the full production pipeline: loading and preprocessing a multi-class image dataset, building a two-phase transfer learning workflow (frozen feature extraction followed by selective fine-tuning), applying data augmentation to handle limited data, training with the complete callback suite, evaluating with confusion matrix and per-class accuracy, and exporting the final model for deployment. By the end you will have a working shot classifier that can be extended to any image classification task by swapping the dataset — this pattern is identical to what Google uses for medical imaging, what IPL franchises use for performance analytics, and what autonomous systems use for object classification.

Analogy🏏Cricket

🏏 Think of it like cricket: Building a neural net from scratch is like a young cricketer learning the game by playing gully cricket with a tape-ball before ever stepping into a coaching academy. You don't use a bowling machine, you don't get video analysis, you don't have a structured training manual — you just play, make mistakes, adapt. Just as gully cricket teaches the core instincts of timing, footwork, and reading the ball that no academy drill can fully replicate, building a network from NumPy teaches the core mechanics of gradient flow, matrix shapes, and numerical stability that no framework hides. Just as every Indian international cricketer traces their instincts back to gully cricket roots, every deep learning practitioner benefits from having once implemented backpropagation themselves. The insight is that frameworks automate what you understand — building from scratch ensures you actually understand it.

Prerequisites

EfficientNetB0 Keras Applications API: include_top=False, weights parameter, pooling='avg'
Transfer learning two-phase pattern: freeze base → train head → unfreeze top layers → fine-tune at 1e-5
Data augmentation layers: RandomFlip, RandomRotation, RandomZoom, RandomBrightness inside model graph
Callbacks: EarlyStopping with restore_best_weights=True, ModelCheckpoint with save_best_only=True
Evaluation metrics: confusion matrix, per-class precision/recall/F1 via classification_report

Setup & Project Structure

This project requires TensorFlow, NumPy, pandas, scikit-learn, and matplotlib. In a real scenario you would load actual cricket shot images; for this exercise we use synthetic image data with the same shape and structure as real RGB frames (224×224×3). The project is structured as a five-step pipeline: data loading and exploration, augmentation and preprocessing pipeline, two-phase model building, training and monitoring, and final evaluation and export. Each step is independent and can be run in a Colab notebook with one code cell per step.

Analogy🏏Cricket

🏏 Think of it like cricket: Before a net session you lay out the ground in clear zones — the bowling machine and pitch (data generation of synthetic IPL stats), the batsman's technique broken into stance, backlift, and follow-through (the NeuralNet forward, backward, and update methods), the throwdown routine that repeats and tracks improvement (the training loop with loss and accuracy monitoring), and finally a match simulation to test readiness (test-set evaluation). Just as a coach keeps the whole drill on one ground so a player can see how each phase connects, keeping all the code in one file lets you trace how a forward pass flows into a gradient and then a weight update. Just as you need only bat, ball, and pitch — not a full stadium — for a productive net, this exercise needs only NumPy, no heavy libraries. The payoff: a clean, well-sectioned practice structure means every mechanic is visible and debuggable, so you understand exactly why the network learns rather than treating it as a black box.

bash

# Setup — install dependencies
# pip install tensorflow scikit-learn matplotlib seaborn

import numpy as np
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers, callbacks
from sklearn.metrics import confusion_matrix, classification_report
from sklearn.model_selection import train_test_split
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import seaborn as sns

np.random.seed(42); tf.random.set_seed(42)
print(f'TF: {tf.__version__} | GPU: {bool(tf.config.list_physical_devices("GPU"))}')

# Shot classes
SHOT_CLASSES = ['cover_drive', 'pull_shot', 'sweep', 'flick', 'straight_drive']
N_CLASSES    = len(SHOT_CLASSES)
IMG_SIZE     = 224
print(f'Classes: {SHOT_CLASSES}')

Step 1 — Foundation

Step 1 generates the synthetic dataset and builds the preprocessing pipeline. We create 5,000 synthetic RGB images (224×224×3) with five class labels. In a real project you would load images from disk using keras.utils.image_dataset_from_directory(). The preprocessing step normalises pixel values to the EfficientNet input range (the model's built-in preprocess_input handles this) and splits into 70/15/15 train/val/test. The key constraint: the StandardScaler-equivalent operation must be fitted only on training data to prevent data leakage.

Analogy🏏Cricket

🏏 Think of it like cricket: Before the first ball is bowled, the groundsman prepares the pitch — the pitch condition is the foundation that determines what is possible. Bad pitch preparation (wrong initialisation) makes even the best bowlers and batsmen ineffective. Just as a pitch that is too green (over-seamed) or too dry (over-spun) constrains the entire match, bad weight initialisation constrains the entire training run. He initialisation is the 'neutral pitch' — well-prepared, giving both batting and bowling a fair contest, from which any outcome is possible.

python

import numpy as np, tensorflow as tf
from sklearn.model_selection import train_test_split
np.random.seed(42)

N_SAMPLES = 5000
IMG_SIZE  = 224
N_CLASSES = 5
SHOT_CLASSES = ['cover_drive', 'pull_shot', 'sweep', 'flick', 'straight_drive']

# Synthetic dataset: random RGB images + class labels
# In real use: tf.keras.utils.image_dataset_from_directory('shots/', label_mode='int')
X = np.random.randint(0, 256, (N_SAMPLES, IMG_SIZE, IMG_SIZE, 3), dtype=np.uint8)
y = np.random.randint(0, N_CLASSES, N_SAMPLES)

print(f'Dataset: {X.shape}, labels: {y.shape}, classes: {N_CLASSES}')
print(f'Class distribution:')
for i, name in enumerate(SHOT_CLASSES):
    print(f'  {name}: {(y==i).sum()} samples ({(y==i).mean():.1%})')

# Split: 70/15/15
X_trainval, X_test, y_trainval, y_test = train_test_split(
    X, y, test_size=0.15, random_state=42, stratify=y)
X_train, X_val, y_train, y_val = train_test_split(
    X_trainval, y_trainval, test_size=0.176, random_state=42, stratify=y_trainval)

print(f'\nSplit — Train: {len(X_train)}, Val: {len(X_val)}, Test: {len(X_test)}')

# Convert to float32 [0,255] — EfficientNet's preprocess_input handles normalisation
X_train = X_train.astype(np.float32)
X_val   = X_val.astype(np.float32)
X_test  = X_test.astype(np.float32)

# tf.data pipelines
BATCH_SIZE = 32
train_ds = (tf.data.Dataset.from_tensor_slices((X_train, y_train))
            .shuffle(3000).batch(BATCH_SIZE).prefetch(tf.data.AUTOTUNE))
val_ds   = (tf.data.Dataset.from_tensor_slices((X_val, y_val))
            .batch(BATCH_SIZE).prefetch(tf.data.AUTOTUNE))
test_ds  = (tf.data.Dataset.from_tensor_slices((X_test, y_test))
            .batch(BATCH_SIZE).prefetch(tf.data.AUTOTUNE))
print('Pipelines ready')

Step 2 — Core Logic

Step 2 builds the complete two-phase transfer learning model with augmentation. Phase 1 trains only the new classification head (base frozen). Phase 2 unfreezes the top 20 layers and fine-tunes at 1e-5. The augmentation pipeline (RandomFlip, RandomRotation, RandomZoom, RandomBrightness) is embedded in the model graph so it activates automatically during training and deactivates at inference. EfficientNet's preprocess_input is also embedded in the model for clean inference API.

Analogy🏏Cricket

🏏 Think of it like cricket: The NeuralNet class is the team management system — it tracks every player's contribution (forward pass caching), attributes match outcomes to specific decisions (backward pass gradients), and adjusts each player's role for the next match (weight update). Just as the team manager must record which batsman faced which bowler and how many runs were scored (cache activations) before attributing success or failure, the neural net must cache z and a at every layer before computing gradients. The momentum in SGD is the team's institutional memory — great past performances inform future selection policy, not overriding current evidence but weighted alongside it.

python

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers, callbacks
import numpy as np

N_CLASSES = 5; IMG_SIZE = 224

def build_phase1_model():
    """Phase 1: Frozen EfficientNetB0 base + trainable head."""
    base = keras.applications.EfficientNetB0(
        include_top=False, weights=None,     # use 'imagenet' for real transfer
        input_shape=(IMG_SIZE, IMG_SIZE, 3), pooling='avg'
    )
    base.trainable = False  # FREEZE all base layers

    inputs = keras.Input(shape=(IMG_SIZE, IMG_SIZE, 3))
    # Augmentation — only active at training=True
    x = layers.RandomFlip('horizontal')(inputs)
    x = layers.RandomRotation(0.15)(x)
    x = layers.RandomZoom(0.1)(x)
    x = layers.RandomBrightness(0.2)(x)
    x = layers.RandomContrast(0.15)(x)
    # EfficientNet preprocessing (normalise to model's expected range)
    x = keras.applications.efficientnet.preprocess_input(x)
    # Feature extraction (frozen)
    x = base(x, training=False)    # training=False: use running BN stats
    # Trainable head
    x = layers.Dense(512, activation='relu')(x)
    x = layers.Dropout(0.4)(x)
    x = layers.Dense(256, activation='relu')(x)
    x = layers.Dropout(0.3)(x)
    outputs = layers.Dense(N_CLASSES, activation='softmax')(x)

    model = keras.Model(inputs, outputs, name='cricket_shot_phase1')
    model.compile(
        optimizer=keras.optimizers.Adam(1e-3),
        loss='sparse_categorical_crossentropy',
        metrics=['accuracy', keras.metrics.SparseTopKCategoricalAccuracy(k=2, name='top2_acc')]
    )
    return model, base

model, base_model = build_phase1_model()
model.summary()
trainable_params = sum(v.numpy().size for v in model.trainable_variables)
print(f'Phase 1 trainable params: {trainable_params:,} (head only)')

Step 3 — Integration & Enhancement

Step 3 executes both training phases with full callback monitoring and plots per-epoch learning curves. Phase 1 trains the head to convergence (typically 10–20 epochs for real data). Phase 2 unfreezes top 20 layers and fine-tunes at 1e-5 for an additional 10–20 epochs. The combined history is plotted as a single continuous learning curve showing both phases — the transition point where phase 2 begins often shows a brief training accuracy dip followed by a val accuracy improvement as the base model adapts to the cricket domain.

Analogy🏏Cricket

🏏 Think of it like cricket: Mini-batch training is like a cricket coaching session where the coach evaluates the batsman against a random set of 32 deliveries from the session's total pool, not all 500 deliveries at once. Evaluating all 500 deliveries before giving any feedback (full-batch) is slow and ignores the fact that early corrections from the first 32 balls can already improve performance on balls 33–64. Mini-batch feedback (stochastic gradients) allows the batsman to improve continuously throughout the session, converging to good technique faster than waiting for the end-of-session review. The randomness (shuffle each epoch) ensures the coach doesn't accidentally train the batsman to handle only one type of delivery sequence.

python

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers, callbacks
import matplotlib.pyplot as plt
import numpy as np

# (Paste data + model from Steps 1 & 2 above)

def get_callbacks(phase, model_path):
    return [
        callbacks.EarlyStopping(
            monitor='val_accuracy', patience=7 if phase==1 else 5,
            restore_best_weights=True, mode='max', verbose=1),
        callbacks.ModelCheckpoint(
            model_path, monitor='val_accuracy', save_best_only=True,
            mode='max', verbose=0),
        callbacks.ReduceLROnPlateau(
            monitor='val_loss', factor=0.5, patience=3, min_lr=1e-7, verbose=1)
    ]

# ── Phase 1: Train head ────────────────────────────────────────
print('=== PHASE 1: Feature Extraction ===')
hist1 = model.fit(
    train_ds, epochs=30,
    validation_data=val_ds,
    callbacks=get_callbacks(1, '/tmp/shot_classifier_p1.keras'),
    verbose=1
)
print(f'Phase 1 best val acc: {max(hist1.history["val_accuracy"]):.2%}')

# ── Phase 2: Unfreeze top 20, fine-tune ───────────────────────
print('\n=== PHASE 2: Fine-Tuning ===')
base_model.trainable = True
for layer in base_model.layers[:-20]:
    layer.trainable = False

model.compile(
    optimizer=keras.optimizers.Adam(1e-5),   # 100x lower LR
    loss='sparse_categorical_crossentropy',
    metrics=['accuracy', keras.metrics.SparseTopKCategoricalAccuracy(k=2, name='top2_acc')]
)
hist2 = model.fit(
    train_ds, epochs=20,
    validation_data=val_ds,
    callbacks=get_callbacks(2, '/tmp/shot_classifier_best.keras'),
    verbose=1
)
print(f'Phase 2 best val acc: {max(hist2.history["val_accuracy"]):.2%}')

# ── Combined learning curves ───────────────────────────────────
def plot_combined_history(h1, h2, title='Cricket Shot Classifier'):
    fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(14, 5))
    e1 = range(1, len(h1.history['loss'])+1)
    e2 = range(len(e1)+1, len(e1)+len(h2.history['loss'])+1)
    ax1.plot(e1, h1.history['loss'],         'b-',  label='Phase1 Train')
    ax1.plot(e1, h1.history['val_loss'],     'b--', label='Phase1 Val')
    ax1.plot(e2, h2.history['loss'],         'r-',  label='Phase2 Train')
    ax1.plot(e2, h2.history['val_loss'],     'r--', label='Phase2 Val')
    ax1.axvline(len(e1), color='gray', linestyle=':', label='Fine-tune start')
    ax1.set_title(f'{title} — Loss'); ax1.set_xlabel('Epoch'); ax1.legend(); ax1.grid(True,alpha=0.3)
    ax2.plot(e1, h1.history['accuracy'],     'b-',  label='Phase1 Train')
    ax2.plot(e1, h1.history['val_accuracy'], 'b--', label='Phase1 Val')
    ax2.plot(e2, h2.history['accuracy'],     'r-',  label='Phase2 Train')
    ax2.plot(e2, h2.history['val_accuracy'], 'r--', label='Phase2 Val')
    ax2.axvline(len(e1), color='gray', linestyle=':', label='Fine-tune start')
    ax2.set_title(f'{title} — Accuracy'); ax2.set_xlabel('Epoch'); ax2.legend(); ax2.grid(True,alpha=0.3)
    plt.tight_layout()
    plt.savefig('/tmp/shot_classifier_curves.png', dpi=150, bbox_inches='tight')
    print('Saved: /tmp/shot_classifier_curves.png')

plot_combined_history(hist1, hist2)

Step 4 — Testing & Verification

Step 4 evaluates the best checkpoint on the held-out test set, generates the confusion matrix, and computes per-class metrics. The confusion matrix reveals which shots the classifier confuses most — for example, the flick and the pull shot share similar bat position and are likely the hardest to distinguish. Per-class precision, recall, and F1-score from classification_report show which shot types are systematically underdetected (low recall) versus which generate false positives (low precision). The model is exported to SavedModel format for deployment.

Analogy🏏Cricket

🏏 Think of it like cricket: Verifying the network is a batsman's fitness test with three non-negotiable checks. First, form must steadily improve — the loss must fall monotonically over the first 200 epochs, just as a player's error count should drop session after session, not swing wildly. Second, real skill must beat luck — test accuracy above 70% when random guessing is 50%, exactly as a batsman must clearly outscore a tail-ender swinging blindly to prove genuine technique. Third, every part of the technique must be engaged — non-zero gradient norms confirm the learning signal actually flows through all three layers, like checking footwork, backlift, and follow-through are each working rather than one part being frozen. The gold standard is the numerical gradient check: nudge a weight a tiny amount and confirm the loss changes as predicted, just as a coach films from two angles to confirm a fix is real, not imagined. The payoff: these checks prove your from-scratch network is genuinely learning and correctly wired, not just accidentally producing a decent number.

bash

import numpy as np, tensorflow as tf
from tensorflow import keras
from sklearn.metrics import confusion_matrix, classification_report
import matplotlib.pyplot as plt
import seaborn as sns

SHOT_CLASSES = ['cover_drive', 'pull_shot', 'sweep', 'flick', 'straight_drive']

# Load best checkpoint and evaluate ONCE on test set
best_model = keras.models.load_model('/tmp/shot_classifier_best.keras')

# Predictions
y_prob = best_model.predict(test_ds, verbose=0)
y_pred = y_prob.argmax(axis=1)

# The test labels from the tf.data pipeline
y_true = np.concatenate([y.numpy() for _, y in test_ds])

# Overall metrics
test_acc = (y_pred == y_true).mean()
print(f'=== TEST SET RESULTS ===')
print(f'Accuracy: {test_acc:.2%}')
print(f'Top-2 Accuracy: {(y_prob.argsort(axis=1)[:,-2:] == y_true.reshape(-1,1)).any(axis=1).mean():.2%}')
print('\nPer-class Report:')
print(classification_report(y_true, y_pred, target_names=SHOT_CLASSES))

# Confusion matrix
cm = confusion_matrix(y_true, y_pred)
fig, ax = plt.subplots(figsize=(8, 7))
sns.heatmap(cm, annot=True, fmt='d', cmap='Blues',
            xticklabels=SHOT_CLASSES, yticklabels=SHOT_CLASSES, ax=ax)
ax.set_xlabel('Predicted Shot'); ax.set_ylabel('Actual Shot')
ax.set_title('IPL Shot Classifier — Confusion Matrix')
plt.xticks(rotation=30, ha='right'); plt.tight_layout()
plt.savefig('/tmp/shot_confusion_matrix.png', dpi=150, bbox_inches='tight')
print('Saved confusion matrix: /tmp/shot_confusion_matrix.png')

# Export for deployment
best_model.export('/tmp/cricket_shot_classifier_v1')
print('Model exported to /tmp/cricket_shot_classifier_v1 (SavedModel format)')

# Quick inference demo
sample_frame = np.random.randint(0, 256, (1, 224, 224, 3), dtype=np.uint8).astype(np.float32)
pred_probs = best_model.predict(sample_frame, verbose=0)[0]
pred_class = SHOT_CLASSES[pred_probs.argmax()]
print(f'\nSample prediction: {pred_class} ({pred_probs.max():.2%} confidence)')
print('Expected test accuracy on synthetic data: ~20-25% (random labels, no signal)')

Warning: On real cricket shot data, if the confusion matrix shows a single off-diagonal cell with very high counts (e.g., pull_shot consistently classified as flick), this indicates systematic feature overlap — these two shots share similar wrist position and body angle features. The fix is targeted data augmentation (add more variety in the confused classes), more training data for those classes, or adding temporal context (optical flow or consecutive frames) which better distinguishes shots that differ primarily in timing.

Extension Challenge: (1) Replace synthetic images with real cricket shot images from a public dataset (e.g., Cricket Shot Recognition datasets on Kaggle) and measure actual transfer learning performance. (2) Add a sixth class 'no_shot' (between-delivery frames) and observe how the classifier handles background frames — this forces the model to distinguish action from non-action. (3) Convert the SavedModel to TensorFlow Lite (tf.lite.TFLiteConverter.from_saved_model()) for deployment on a mobile app that classifies shots in real time from a phone camera.

Build augmentation inside the model graph with Keras layers — they auto-activate during training and deactivate at inference, eliminating the need for separate augment/no-augment pipelines.
Phase 1 trains only the Dense head (base.trainable=False) with LR=1e-3; Phase 2 unfreezes top 20 base layers and fine-tunes at LR=1e-5 — always re-compile after changing trainable.
Always call base_model(x, training=False) in the forward pass — frozen BatchNorm must use its stored running statistics, not compute new ones from your task dataset.
The confusion matrix reveals systematic misclassifications between visually similar classes — pull_shot vs flick confusion is actionable: add temporal context or collect more discriminative examples.
Export the final model with model.export() (SavedModel format) for production deployment — SavedModel includes preprocessing, augmentation (deactivated), and the full inference graph.
Evaluate on the test set exactly once after all hyperparameter decisions — multiple evaluations turn the test set into implicit validation, inflating measured performance.