Lesson 35 is the final Capstone submission: deploying the optimised model from Lesson 34, evaluating it on the sealed test set for the first and only time, and submitting a complete project report documenting every decision from data pipeline to deployment. This is the most important lesson in Course 5 — it synthesises all 34 preceding lessons into one unified, documented, production-quality deep learning project. The deployment step produces a deployable model artefact: a TensorFlow SavedModel with a prediction function (Option A) or a HuggingFace pipeline exported to the Hub (Option B). The test set evaluation is the final, unbiased performance measurement — the number that appears in the Capstone report as the headline result. The project report documents the complete project lifecycle in six sections: dataset, architecture justification, training protocol, optimisation experiments, final evaluation, and deployment. A strong Capstone report is as valuable as the model itself — it demonstrates the ability to communicate engineering decisions clearly to collaborators, stakeholders, and future maintainers.
Analogy🏏Cricket
🏏 Think of it like cricket: Lesson 35 is the IPL final — the training is over, the squad is selected, and the match must be played against the opposition (test set) for the first and only time under real tournament conditions. There are no warm-up balls, no second chances; the pre-match preparation (Lessons 31–34) must speak for itself. The project report is the post-match press conference and official scorecard submission — the team must explain every tactical decision to the media and the ICC (evaluators), not just announce the final score. A team that won by 6 wickets but cannot explain why they chose that batting order, or what adjustments they made at the halfway stage, fails the full assessment even if the match result was positive. The deployment artefact is the team's kit and formation exported for the franchise's next match — proven, documented, ready for immediate use by any member of the coaching staff.
Learning Objectives
- Evaluate on sealed test set — report final unbiased performance metrics (accuracy/weighted F1) as the headline Capstone result
- Deploy Option A to TensorFlow SavedModel with a prediction function or TFLite for mobile; Option B to HuggingFace pipeline format
- Write a 6-section Capstone project report covering dataset, architecture, training, optimisation, evaluation, and deployment
- Demonstrate production ML engineering discipline: no test set leakage, systematic optimisation, documented decision rationale
- Optionally upload the trained model to the HuggingFace Hub or Google Drive for portfolio visibility
Technical Requirements
- Option A: ≥75% test accuracy on the sealed test set; TFLite export and an inference function accepting a raw image path
- Option B: ≥80% weighted F1 on the sealed test set; HuggingFace pipeline() inference demo with 5 sample commentaries
- Per-class precision, recall, and F1 for ALL classes reported in the project report (not just macro averages)
- Training curves showing loss and accuracy/F1 for both phases (Option A) or full fine-tuning run (Option B)
- Complete experiment comparison table from Phase 3 showing before/after metrics for each optimisation
- Confusion matrix annotated with class names for both Phase 2 baseline and Phase 3 final (two matrices in report)
Architecture & Design
The Capstone report has six mandatory sections. Section 1 — Dataset: class counts per split, class distribution histogram, augmentation strategy, leakage prevention checklist. Section 2 — Architecture: model choice with justification (why EfficientNetB0 over ResNet50, why DistilBERT over BERT-base), parameter count, trainable parameter count per phase, and comparison to the discarded alternative. Section 3 — Training: training curves for both phases, callback configuration, learning rate schedule, and explanation of any early stopping events. Section 4 — Optimisation: experiment table (change, before-metric, after-metric), confusion matrix comparison (Phase 2 vs Phase 3), and interpretation of why each optimisation worked or did not work. Section 5 — Final Evaluation: test set results with per-class report and confusion matrix, comparison against baseline (random chance and zero-shot for Option B), and analysis of remaining errors. Section 6 — Deployment: code listing of the inference function, latency measurement, model size, and instructions for reproducing the full project from scratch.
Analogy🏏Cricket
🏏 Think of it like cricket: The six-section project report is the complete IPL franchise season documentation: Section 1 (Dataset) = squad acquisition report — who was recruited, how many of each role, fitness status. Section 2 (Architecture) = tactical formation justification — why this batting order, why this bowling strategy, what alternatives were considered. Section 3 (Training) = match-by-match season progression — performance trends across the full season. Section 4 (Optimisation) = mid-season tactical adjustments — what changed, why, and what improved. Section 5 (Final Evaluation) = qualification/final match performance report — actual results against the opposition. Section 6 (Deployment) = franchise operations manual — how any new coaching staff can replicate the team's success in the next season. A franchise document missing any of these sections is incomplete regardless of the final match result.
python
import numpy as np, tensorflow as tf, torch
from tensorflow import keras
from transformers import pipeline as hf_pipeline
from sklearn.metrics import classification_report, confusion_matrix
import matplotlib.pyplot as plt, matplotlib; matplotlib.use('Agg')
import seaborn as sns, time
np.random.seed(42)
print('CAPSTONE PHASE 4 — DEPLOYMENT AND FINAL EVALUATION')
print('=' * 60)
# ══════════════════════════════════════════════════════════
# OPTION A: Load final model + test set evaluation (ONCE)
# ══════════════════════════════════════════════════════════
print('\n[A] Loading optimised model...')
final_model_A = keras.models.load_model('/tmp/capstone_A_final.keras')
print('[A] Test set evaluation (SEALED SET — FIRST AND ONLY TIME)')
y_pred_test_A = final_model_A.predict(test_ds_A, verbose=0).argmax(axis=1)
y_true_test_A = np.concatenate([y.numpy() for _, y in test_ds_A])
test_acc_A = (y_pred_test_A == y_true_test_A).mean()
print(f'\n[A] FINAL TEST ACCURACY: {test_acc_A:.2%}')
print(f' Threshold: 75% Status: {"✓ PASSED" if test_acc_A >= 0.75 else "✗ BELOW THRESHOLD"}')
print('\nPer-class Report (TEST SET):')
print(classification_report(y_true_test_A, y_pred_test_A, target_names=SHOT_CLASSES))
# Test confusion matrix
cm_test_A = confusion_matrix(y_true_test_A, y_pred_test_A)
fig,ax=plt.subplots(figsize=(9,7))
sns.heatmap(cm_test_A,annot=True,fmt='d',cmap='Blues',
xticklabels=SHOT_CLASSES,yticklabels=SHOT_CLASSES,ax=ax)
ax.set_title(f'FINAL TEST CONFUSION MATRIX — Accuracy: {test_acc_A:.2%}')
plt.xticks(rotation=30,ha='right'); plt.tight_layout()
plt.savefig('/tmp/capstone_A_TEST_confusion.png',dpi=150,bbox_inches='tight')
print('Final test confusion matrix saved.')
# ══════════════════════════════════════════════════════════
# OPTION B: Load final model + test set evaluation (ONCE)
# ══════════════════════════════════════════════════════════
print('\n[B] Loading optimised DistilBERT model...')
sentiment_inference = hf_pipeline(
'text-classification',
model='/tmp/capstone_B_final',
tokenizer='/tmp/capstone_B_final',
device=-1
)
print('[B] Test set evaluation (SEALED SET — FIRST AND ONLY TIME)')
preds_test_B, labels_test_B = [], []
for batch in torch.utils.data.DataLoader(test_ds_B, batch_size=32):
with torch.no_grad():
out = weighted_trainer.model(
input_ids=batch['input_ids'],
attention_mask=batch['attention_mask']
)
preds_test_B.extend(out.logits.argmax(dim=-1).cpu().numpy())
labels_test_B.extend(batch['labels'].cpu().numpy())
from sklearn.metrics import f1_score
test_f1_B = f1_score(labels_test_B, preds_test_B, average='weighted')
test_acc_B = np.mean(np.array(preds_test_B) == np.array(labels_test_B))
print(f'\n[B] FINAL TEST WEIGHTED F1: {test_f1_B:.3f}')
print(f' TEST ACCURACY: {test_acc_B:.2%}')
print(f' Threshold: 0.80 Status: {"✓ PASSED" if test_f1_B >= 0.80 else "✗ BELOW THRESHOLD"}')
print('\nPer-class Report (TEST SET):')
print(classification_report(labels_test_B, preds_test_B, target_names=EVENT_CLASSES))
cm_test_B = confusion_matrix(labels_test_B, preds_test_B)
fig,ax=plt.subplots(figsize=(10,8))
sns.heatmap(cm_test_B,annot=True,fmt='d',cmap='Blues',
xticklabels=EVENT_CLASSES,yticklabels=EVENT_CLASSES,ax=ax)
ax.set_title(f'FINAL TEST CONFUSION MATRIX — Weighted F1: {test_f1_B:.3f}')
plt.xticks(rotation=30,ha='right'); plt.tight_layout()
plt.savefig('/tmp/capstone_B_TEST_confusion.png',dpi=150,bbox_inches='tight')
print('Final test confusion matrix saved.')Phase 1 — Core Implementation
Phase 1 of deployment builds the inference API — the function that any external caller can use to get predictions from the trained model without understanding its internals. A good inference API accepts raw input (image file path or raw text string), handles all preprocessing internally, returns a structured result (class label, confidence, all class probabilities), and includes latency measurement.
Analogy🏏Cricket
🏏 Think of it like cricket: The inference API is the IPL franchise's public-facing scouting report service — any team can query 'what shot type is this player most likely playing in this clip?' and get a clear, structured answer without understanding the underlying analytics model. The API hides the complexity (preprocessing, model forward pass, postprocessing) just as the scouting report service hides the video analysis infrastructure. The caller needs only to submit a clip and receive 'cover drive: 87% confidence' — the engineering plumbing is internal.
python
import numpy as np, tensorflow as tf, torch, time
from tensorflow import keras
from transformers import pipeline as hf_pipeline
# ══════════════════════════════════════════════════════════
# OPTION A: Production inference function
# ══════════════════════════════════════════════════════════
class IPLShotClassifier:
"""Production inference API for IPL shot type classification."""
CLASSES = ['cover_drive','pull_shot','sweep','flick','straight_drive','helicopter']
def __init__(self, model_path='/tmp/capstone_A_final.keras'):
self.model = keras.models.load_model(model_path)
self.model(np.zeros((1,224,224,3), dtype=np.float32), training=False) # warmup
def predict(self, image_array):
"""
Args: image_array — numpy array (H, W, 3), uint8 [0,255]
Returns: {'shot_type': str, 'confidence': float, 'all_probs': dict}
"""
img = tf.image.resize(image_array, (224,224)).numpy()
img = img[np.newaxis].astype(np.float32) # add batch dim
t0 = time.time()
probs = self.model(img, training=False).numpy()[0]
latency_ms = (time.time() - t0) * 1000
top_idx = probs.argmax()
return {
'shot_type': self.CLASSES[top_idx],
'confidence': float(probs[top_idx]),
'all_probs': {c: float(p) for c, p in zip(self.CLASSES, probs)},
'latency_ms': round(latency_ms, 1)
}
def predict_batch(self, image_arrays):
imgs = np.stack([tf.image.resize(img,(224,224)).numpy() for img in image_arrays])
probs = self.model(imgs.astype(np.float32), training=False).numpy()
return [{'shot_type': self.CLASSES[p.argmax()], 'confidence': float(p.max())}
for p in probs]
# Demo inference
classifier_A = IPLShotClassifier()
test_frame = np.random.randint(0, 256, (480, 854, 3), dtype=np.uint8) # typical HD frame
result = classifier_A.predict(test_frame)
print(f'[A] Prediction: {result["shot_type"]} ({result["confidence"]:.1%} confidence)')
print(f' Latency: {result["latency_ms"]}ms')
print(f' All probs: {result["all_probs"]}')
# ══════════════════════════════════════════════════════════
# OPTION B: Production inference function
# ══════════════════════════════════════════════════════════
commentary_pipe = hf_pipeline(
'text-classification',
model='/tmp/capstone_B_final',
tokenizer='/tmp/capstone_B_final',
device=-1, top_k=None # return all class probabilities
)
def classify_commentary(text):
"""Classify a cricket commentary string into an IPL event type."""
t0 = time.time()
results = commentary_pipe(text[:512])[0] # truncate at 512 chars
latency_ms = (time.time() - t0) * 1000
top = max(results, key=lambda x: x['score'])
return {
'event_type': top['label'],
'confidence': float(top['score']),
'all_probs': {r['label']: float(r['score']) for r in results},
'latency_ms': round(latency_ms, 1)
}
# Demo inference
test_commentaries = [
'Rohit Sharma brings up his century with a glorious six',
'Bumrah bowls a perfect yorker to take the crucial wicket',
'The drinks break is taken at the end of the 10th over',
]
print('\n[B] Commentary Classification Demo:')
for text in test_commentaries:
r = classify_commentary(text)
print(f' [{r["event_type"]:22s} {r["confidence"]:.0%}] {text[:55]}')Phase 2 — Feature Completion
Phase 2 adds TFLite export (Option A) for mobile deployment, model size measurement, and the HuggingFace Hub upload template (Option B). TFLite converts the TensorFlow SavedModel to a compressed, optimised format that runs on Android and iOS devices — reducing model size by 50–75% through quantisation while maintaining most accuracy. The Hub upload makes the model publicly accessible and provides a model card that documents the model for external users.
Analogy🏏Cricket
🏏 Think of it like cricket: TFLite export is the team's travel kit — the same players and tactics compressed into a lightweight format that fits in a carry-on bag for away matches (mobile deployment), rather than the full training facility required at home. Mobile deployment allows the shot classifier to run on a smartphone in the stadium without internet connectivity — a crucial capability for on-field use during the match. The HuggingFace Hub upload is publishing the team's analytical playbook on the official ICC platform — any franchise in the world can now access and build on your work, accelerating the entire cricket analytics ecosystem.
python
import tensorflow as tf, os
from tensorflow import keras
import numpy as np
# ══════════════════════════════════════════════════════════
# OPTION A: TFLite Export
# ══════════════════════════════════════════════════════════
print('[A] Exporting to TFLite...')
# Save as SavedModel first
final_model_A.export('/tmp/capstone_A_savedmodel')
# Convert to TFLite with float16 quantisation
converter = tf.lite.TFLiteConverter.from_saved_model('/tmp/capstone_A_savedmodel')
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_types = [tf.float16] # float16 quantisation
tflite_model = converter.convert()
tflite_path = '/tmp/capstone_A_shot_classifier.tflite'
with open(tflite_path, 'wb') as f:
f.write(tflite_model)
original_size_mb = os.path.getsize('/tmp/capstone_A_final.keras') / (1024**2)
tflite_size_mb = os.path.getsize(tflite_path) / (1024**2)
print(f' SavedModel size: ~{original_size_mb:.1f} MB')
print(f' TFLite size: {tflite_size_mb:.1f} MB (float16 quantisation)')
print(f' Compression: {original_size_mb/max(tflite_size_mb,0.1):.1f}x smaller')
# TFLite inference demo
interpreter = tf.lite.Interpreter(model_path=tflite_path)
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
test_input = np.zeros((1,224,224,3), dtype=np.float32)
interpreter.set_tensor(input_details[0]['index'], test_input)
interpreter.invoke()
tflite_output = interpreter.get_tensor(output_details[0]['index'])
print(f' TFLite inference OK: output shape {tflite_output.shape}')
print(f' Predicted class: {SHOT_CLASSES[tflite_output.argmax()]}')
# ══════════════════════════════════════════════════════════
# OPTION B: HuggingFace Hub Upload Template
# ══════════════════════════════════════════════════════════
print('\n[B] HuggingFace Hub upload template:')
hub_template = '''
# Uncomment and run with valid HuggingFace token:
# from huggingface_hub import HfApi, login
# login(token='your_hf_token') # get from huggingface.co/settings/tokens
#
# api = HfApi()
# api.create_repo('your-username/ipl-commentary-event-classifier', private=False)
#
# from transformers import AutoModelForSequenceClassification, AutoTokenizer
# model = AutoModelForSequenceClassification.from_pretrained('/tmp/capstone_B_final')
# tokenizer = AutoTokenizer.from_pretrained('/tmp/capstone_B_final')
# model.push_to_hub('your-username/ipl-commentary-event-classifier')
# tokenizer.push_to_hub('your-username/ipl-commentary-event-classifier')
#
# Then anyone can use it with:
# pipe = pipeline('text-classification', model='your-username/ipl-commentary-event-classifier')
# result = pipe('Rohit hit a six to win the match!')
'''
print(hub_template)Phase 3 — Polish & Production Readiness
Phase 3 assembles the complete project report as a structured document. The report is the final deliverable — it is what evaluators assess the Capstone against. A strong report demonstrates: (1) understanding of why each decision was made, not just what was done; (2) honest reporting of failures and what was learned from them; (3) quantitative evidence for every claim (not 'the model performed well' but 'the model achieved 82% test accuracy vs 16.7% random baseline'); (4) awareness of the deployed model's limitations and failure modes.
Analogy🏏Cricket
🏏 Think of it like cricket: The Capstone report is the post-tournament dossier a captain submits to the board — it is what the season is judged on, not the individual highlights. A weak report just lists scores; a strong one, like a great captain's review, does three things. First, it explains why each decision was made, not merely what happened — why this bowler at the death, why this batting order — showing understanding, not luck. Second, it reports failures honestly and what was learned from them, just as an honest captain owns the overs that went for plenty rather than hiding them. Third, it backs every claim with numbers — not 'the side batted well' but exact run rates, strike rates, and win margins — just as your report must give quantitative evidence for every performance claim rather than vague praise. Just as a board rates a captaincy on the quality of reasoning and evidence, evaluators assess the Capstone on the report. The payoff: a reasoned, honest, evidence-backed write-up is what turns a working model into a credible, production-ready, defensible deliverable.
python
# Capstone Project Report Template
# Complete each section with your specific project details
REPORT_TEMPLATE = '''
================================================================
SKILLVERIS COURSE 5 CAPSTONE PROJECT REPORT
Option [A/B]: [Image/Text] Classification
Project: [IPL Shot Type Classifier / IPL Commentary Event Tagger]
================================================================
SECTION 1 — DATASET
─────────────────────
Total samples: [N]
Classes: [list all classes]
Split: Train=[N_tr] / Val=[N_val] / Test=[N_te] (stratified)
Class distribution (all splits): [table]
Augmentation strategy: [list augmentations and rationale]
Leakage prevention: [describe what was fitted on training only]
SECTION 2 — ARCHITECTURE JUSTIFICATION
────────────────────────────────────────
Chosen architecture: [EfficientNetB0 / DistilBERT]
Justification: [3-4 sentences explaining why — cite params, pre-training, dataset size]
Alternative considered: [ResNet50 / BERT-base]
Reason rejected: [parameter count, training time, accuracy trade-off]
Total parameters: [N]
Phase 1 trainable: [N] ([%])
Phase 2 trainable: [N] ([%])
SECTION 3 — TRAINING
─────────────────────
Optimiser: Adam, lr=[1e-3/1e-5/2e-5], warmup=[yes/no]
Callbacks: EarlyStopping (patience=[N]), ModelCheckpoint, ReduceLROnPlateau
Phase 1 epochs: [N] (stopped at epoch [N])
Phase 1 best val: [metric]=[value]
Phase 2 epochs: [N] (stopped at epoch [N])
Phase 2 best val: [metric]=[value]
[Attach training curves: /tmp/capstone_A/B_curves.png]
SECTION 4 — OPTIMISATION EXPERIMENTS
──────────────────────────────────────
| Experiment | Change | Before | After | Delta |
|---------------------|---------------------------|----------|----------|--------|
| Phase 2 Baseline | Two-phase fine-tuning | [metric] | - | - |
| Exp 1: class weights| compute_class_weight | [before] | [after] | [Δ] |
| Exp 2: [your change]| [description] | [before] | [after] | [Δ] |
| Exp 3: [your change]| [description] | [before] | [after] | [Δ] |
[Attach Phase 2 confusion matrix and Phase 3 final confusion matrix]
SECTION 5 — FINAL EVALUATION (TEST SET)
─────────────────────────────────────────
Final test accuracy: [value]
Final weighted F1: [value] (Option B only)
Random baseline: [1/n_classes * 100]%
Zero-shot baseline: [value]% (Option B only)
Improvement over random: +[N]pp
[Attach final test confusion matrix]
Per-class performance:
[Paste classification_report output here]
Remaining error analysis:
- Largest off-diagonal: [class X] misclassified as [class Y] ([N] cases)
Hypothesis: [why does this confusion occur?]
- Second largest: [class A] misclassified as [class B] ([N] cases)
Hypothesis: [why?]
SECTION 6 — DEPLOYMENT
────────────────────────
Deployment format: [TFLite / HuggingFace pipeline()]
Model size: [N] MB (full), [N] MB (quantised)
Inference latency: [N]ms per sample on CPU
Reproduction: [list all commands needed to reproduce from scratch]
Limitations: [honest statement of what the model cannot do well]
================================================================
Submitted by: Sri Hayavadhana Info-Tech
Developer: Nagarajarao C R
Platform: SkillVeris
================================================================
'''
with open('/tmp/capstone_report_template.txt', 'w') as f:
f.write(REPORT_TEMPLATE)
print('Report template saved: /tmp/capstone_report_template.txt')
print('Complete each section with your specific project results and submit.')Evaluation Rubric
- Data pipeline quality (25pts): correct stratified splitting, no leakage, preprocessing fitted on train only, class distribution documented with imbalance strategy noted
- Architecture justification (25pts): specific reasoning for architecture choice, comparison to at least one rejected alternative, parameter counts for each training phase documented
- Training discipline (25pts): both phases trained with appropriate callbacks, training curves included, early stopping events documented, hyperparameter choices explained
- Evaluation rigour (15pts): per-class precision/recall/F1 for all classes, test set confusion matrix, comparison to random baseline and zero-shot baseline (Option B)
- Deployment quality (10pts): working inference function with latency measurement, model size reported, TFLite export (A) or Hub upload (B), reproduction instructions complete
Extension Challenges: (1) Implement a confidence-threshold routing system: predictions above 90% confidence are automatically accepted; predictions between 60–90% are flagged for human review; predictions below 60% are sent to the zero-shot classifier as a fallback. Document the precision-recall tradeoff of each confidence tier. (2) Measure inference latency across batch sizes 1, 4, 16, 64 and plot throughput (samples/sec) vs latency — this is the deployment optimisation analysis that production engineering teams conduct before scaling to high-traffic services. (3) Implement model compression: try int8 quantisation (TFLite) for Option A and compare accuracy, size, and latency against float16 — quantise-aware training typically recovers 90–95% of float32 accuracy at 4× size reduction.