Capstone: Production MLOps Platform

This capstone is the culmination of the entire course: you will design and build a production MLOps platform for a cricket analytics organisation, integrating every capability the seven modules covered into one coherent, self-service, governed system. The platform will version data, run tracked training pipelines, promote models through a registry, serve them with autoscaling and safe rollout, monitor them for drift and fairness, expose explanations and security defences, run on infrastructure as code with CI/CD, and present all of it to data scientists through a self-service interface with paved roads and governance baked in. Rather than demonstrating any single technique, you will architect the whole lifecycle as an integrated product, the defining skill of a senior MLOps engineer. By the end you will have the blueprint and core implementation of a platform that takes a cricket dataset all the way to a monitored, explained, defended, self-renewing production model, the concrete embodiment of everything the course has taught and a portfolio centrepiece that proves you can build MLOps as a system, not just use its parts.

Analogy🏏Cricket

🏏 Think of it like cricket: running an international team is not about excelling at any single function, scouting, selection, coaching, match tactics, or post-match review, but about wiring them into one continuous operation where each feeds the next across a whole season. Just as a great cricket operation connects scouting into selection into coaching into match play into review and back into scouting, your pipeline connects data versioning into tracking into registry into serving into monitoring and back into retraining. Just as a team that does each function brilliantly but never links them wins nothing, MLOps tools used in isolation deliver no production system. Just as the season's loop is what sustains performance, the pipeline's loop is what sustains the model. The insight is that mastery is demonstrated by integration, making the parts work together as a living system, not by polishing any part alone.

Learning Objectives

Architect a complete production MLOps platform that integrates data versioning, training, registry, serving, monitoring, governance, and CI/CD into one self-service system.
Connect the lifecycle so an artifact flows with full lineage from raw data through training and promotion to a monitored, explained, defended production endpoint.
Provide self-service paved roads with versioning, monitoring, rollback, fairness checks, and security wired in by default, plus escape hatches for exceptions.
Close the loop with monitoring-driven retraining and safe rollout, so the platform keeps models current and promotes them through shadow and canary stages.
Run the platform on infrastructure as code with CI/CD gates, treating both models and infrastructure as reviewed, reproducible, version-controlled assets.
Reason about and demonstrate the handoffs, governance, and multi-tenancy that turn a collection of MLOps tools into a coherent, productive, accountable platform.

Technical Requirements

A versioned cricket dataset and feature definitions, with each training run referencing a specific data version for reproducibility and lineage.
A tracked training pipeline (orchestrator + experiment tracking) logging params, data version, metrics, and the model artifact for every run.
A model registry with staging and production stages, where promotion compares a candidate against the incumbent on a recent evaluation slice and requires approval for production.
A serving layer that loads the production model by stage reference, validates inputs, autoscales, supports canary rollout, and logs predictions.
A monitoring layer detecting data drift and tracking fairness across a protected group, emitting signals that gate promotion and trigger retraining.
A governance layer attaching explanations to predictions, defending the API (rate limiting, output coarsening, input validation), and recording an audit trail.
Infrastructure as code provisioning the platform and CI/CD pipelines gating model and infrastructure changes, with self-service interfaces presenting it all in ML terms.
Reproducibility and observability throughout: fixed seeds, recorded data versions, and metrics exposed for dashboards and alerts.

Architecture & Design

The platform is organised as a control plane over the integrated lifecycle, presenting self-service interfaces to users while composing the underlying components and enforcing governance. At the bottom sits infrastructure provisioned by code: a Kubernetes cluster, object storage, and databases, all defined in Terraform. On it run the lifecycle services, a data layer with versioned datasets and feature definitions, an orchestrator running tracked training pipelines, a model registry holding versions and stages with lineage, a serving layer exposing autoscaling endpoints with canary support, a monitoring stack computing drift and fairness, and a governance layer providing explanations, API defences, and audit logging. Above these, a control plane integrates them: when a user requests training or deployment through the self-service SDK, it provisions the right resources, wires monitoring and lineage automatically, and enforces policy, registry promotion, fairness gates, approval for production, as paved-road defaults. CI/CD pipelines gate every model and infrastructure change. The components communicate only through stable artifacts, a data version, a run ID, a registry stage, a prediction log, a drift verdict, so lineage and governance hold at every boundary. The critical design principle is that the platform's value is integration and abstraction: users think in datasets, models, and deployments, while the control plane handles the Kubernetes, the wiring, and the policy underneath, with escape hatches for genuine exceptions, turning the whole course's capabilities into one coherent, governed, self-service product.

Analogy🏏Cricket

🏏 Think of it like cricket: a national setup runs as distinct departments, scouting, selection, coaching, match management, analytics, that interact only through defined handoffs: scouting files reports, selectors issue a squad, coaches receive that squad, match management fields the eleven, analysts file a review. Just as each department hands the next a defined artifact (a report, a squad sheet, a review) rather than meddling in its work, each pipeline component hands over a defined artifact (a data version, a run ID, a registry stage). Just as a clear handoff lets the coaching staff change methods without disrupting selection, stable interfaces let serving change without disrupting training. Just as the review feeds back into the next scouting cycle, monitoring feeds back into retraining. The insight is that a robust operation is built from well-bounded units joined by clean handoffs, which is exactly how the pipeline's components connect.

python

# Capstone architecture: the integrated lifecycle behind a self-service control plane.
# Project structure:
# cricket-mlops-platform/
# |-- infra/              Terraform: cluster, storage, DBs (IaC, lesson 22)
# |-- platform/
# |   |-- data.py         versioned datasets + features (lessons 3-4)
# |   |-- pipeline.py     tracked training pipeline (lessons 2,6,7)
# |   |-- registry.py     model registry: stages + lineage (lesson 12)
# |   |-- serving.py      autoscaling endpoints + canary (lessons 13,15,18,23)
# |   |-- monitoring.py   drift + fairness signals (lessons 16,17,27)
# |   |-- governance.py   explanations + API defence + audit (lessons 26,29,30)
# |   `-- control_plane.py integrates all of the above + enforces policy (lesson 34)
# |-- .github/workflows/  CI/CD gates for models + infra (lessons 21,25)
# `-- sdk.py              self-service interface in ML terms (lesson 34)
from typing import Protocol, Optional

# The stable handoffs (artifacts) between components -- the integration contract.
class DataLayer(Protocol):
    def training_set(self, data_version: str): ...     # -> (X, y)
    def reference_features(self): ...                  # baseline for drift/fairness
class Pipeline(Protocol):
    def train_and_track(self, data_version: str, config: dict) -> str: ...  # -> run_id
class Registry(Protocol):
    def promote_if_better(self, run_id: str, eval_set) -> str: ...          # -> stage
    def load_production(self): ...
class Monitoring(Protocol):
    def drift_detected(self) -> bool: ...
    def fairness_ok(self, eval_set, group) -> bool: ...
class Governance(Protocol):
    def explain(self, x): ...
    def guard(self, client_id, x): ...                 # rate-limit + validate + audit
# control_plane.py composes these behind one self-service SDK and enforces policy.
print('Five lifecycle services + a control plane = an integrated, governed platform.')

Phase 1 — Core Implementation

Phase 1 builds the offline lifecycle spine: the data layer, the tracked training pipeline, and the registry with comparison-and-fairness-gated promotion. This is the core because it establishes the lineage and governance backbone, a versioned dataset feeding tracked runs feeding a registry that will not promote a model unless it both beats the incumbent and passes a fairness check. You will implement deterministic data and features, a trainer that logs lineage to the tracker, and a registry whose promotion gate combines accuracy comparison with the fairness gate from Module 6. Getting this phase right means every model is traceable to its data and run and cannot reach production while exhibiting disparate impact, the accountable foundation the rest of the platform depends on.

Analogy🏏Cricket

🏏 Think of it like cricket: before a season you establish the backbone of the operation, a fixed player database, a recorded selection process, and a squad list that traces every pick back to the trials that justified it. Just as the player database is your versioned data and the recorded selection is your tracked training, the traceable squad list is your registry. Just as a squad with no record of why each player was picked cannot be defended or revised, a model with no lineage cannot be audited or rolled back. Just as this backbone underpins every match decision that follows, this offline spine underpins serving and monitoring. The insight is that establishing a traceable selection backbone first is what makes everything downstream accountable and revisable.

python

# Phase 1: data + tracked pipeline + accuracy-AND-fairness-gated registry.
import mlflow, mlflow.sklearn, random, numpy as np
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import accuracy_score

SEED = 1983; MODEL = 'PlayerFormClassifier'
mlflow.set_experiment('cricket-mlops-capstone')

class DataLayer:
    data_version = '[email protected]'
    def __init__(self):
        rng = np.random.default_rng(SEED)
        self.X = rng.uniform([10,60,0.1,8],[65,160,0.75,60], size=(600,4))
        self.group = rng.integers(0,2,600)                 # protected attribute
        self.y = ((0.03*self.X[:,0]+0.02*self.X[:,1]+1.5*self.X[:,2]) > 4.5).astype(int)
    def training_set(self): return self.X, self.y
    def reference_features(self): return self.X[:,0]

def train_and_track(data: DataLayer, max_depth=4) -> str:
    X, y = data.training_set()
    with mlflow.start_run() as run:
        mlflow.log_param('data_version', data.data_version)    # lineage
        mlflow.log_param('max_depth', max_depth)
        m = DecisionTreeClassifier(max_depth=max_depth, random_state=SEED).fit(X, y)
        mlflow.log_metric('accuracy', accuracy_score(y, m.predict(X)))
        mlflow.sklearn.log_model(m, 'model')
        return run.info.run_id

def fairness_ok(model, X, y, group, max_gap=0.15):
    pred = model.predict(X)
    tpr = []
    for g in np.unique(group):
        mask = (group==g) & (y==1)
        tpr.append(pred[mask].mean() if mask.any() else 0.0)
    return (max(tpr)-min(tpr)) <= max_gap     # equal-opportunity gate (Module 6)

def promote_if_better(run_id, data: DataLayer):
    X, y = data.training_set()
    cand = mlflow.sklearn.load_model(f'runs:/{run_id}/model')
    if not fairness_ok(cand, X, y, data.group):
        return 'BLOCKED: fails fairness gate'           # governance gate
    try:
        prod = mlflow.sklearn.load_model(f'models:/{MODEL}/Production')
        if accuracy_score(y, cand.predict(X)) <= accuracy_score(y, prod.predict(X)):
            return 'HOLD: not better than incumbent'
    except Exception:
        pass                                            # first model auto-promotes
    mv = mlflow.register_model(f'runs:/{run_id}/model', MODEL)
    mlflow.MlflowClient().transition_model_version_stage(
        MODEL, mv.version, 'Staging')                    # to Staging, not straight to prod
    return f'PROMOTED v{mv.version} to Staging (passed accuracy + fairness)'

Phase 2 — Feature Completion

Phase 2 adds the online and observability halves: serving the production model by stage with prediction logging and explanations, and monitoring that stream for drift and fairness. This completes the feature set by connecting the offline spine to live traffic, governance, and observation, serving turns the registered model into a defended, explained endpoint, and monitoring turns its prediction log into drift and fairness signals. You will build a serving layer that loads by stage reference, guards the API, attaches a SHAP explanation to each prediction, and logs every decision, plus a monitor that compares the live feature window against the training reference and recomputes fairness on audited decisions. This phase makes the model observable, explainable, and defended in production, the prerequisite for closing the loop.

Analogy🏏Cricket

🏏 Think of it like cricket: with the squad selected, you now field the team and station analysts to watch every ball, the match management puts the chosen eleven into play, and the analysts log each delivery against the expected patterns. Just as match management fields whoever currently holds each role rather than a fixed name, serving loads whoever holds the Production stage rather than a fixed file. Just as the analysts logging every ball create the record that later reveals a developing weakness, prediction logging creates the record the monitor reads for drift. Just as fielding without observation would leave problems unseen, serving without monitoring would leave drift invisible. The insight is that putting the model into play and watching it closely are two halves of the same step, neither complete without the other.

python

# Phase 2: defended + explained serving by stage, with drift + fairness monitoring.
import mlflow, numpy as np, time, json, os
from collections import defaultdict, deque
from scipy import stats
import shap

MODEL = 'PlayerFormClassifier'; AUDIT = 'audit/decisions.log'
FEATURES = ['batting_average','strike_rate','boundary_pct','away_average']

class ServingLayer:
    def __init__(self, reference_features):
        self.model = mlflow.sklearn.load_model(f'models:/{MODEL}/Production')  # by STAGE
        self.explainer = shap.TreeExplainer(self.model)
        self.ref = np.asarray(reference_features)
        self.history = defaultdict(lambda: deque(maxlen=100))
    def _guard(self, client_id, x):                       # security (Module 6)
        now = time.time(); h = self.history[client_id]
        while h and now-h[0] > 1.0: h.popleft()
        h.append(now)
        if len(h) > 5: return 'rate_limited'
        lo, hi = self.ref.min(), self.ref.max()
        if x[0] < lo-0.2*(hi-lo) or x[0] > hi+0.2*(hi-lo): return 'out_of_distribution'
        return None
    def predict(self, client_id, x):
        block = self._guard(client_id, np.asarray(x))
        if block: return {'error': block}
        pred = int(self.model.predict([x])[0])
        sv = np.ravel(self.explainer.shap_values([x]))
        reasons = sorted(zip(FEATURES, sv), key=lambda t: -abs(t[1]))[:3]
        result = {'in_form': bool(pred),                  # coarse output (anti-extraction)
                  'top_reasons': [{'f': f, 'c': round(float(c),3)} for f,c in reasons]}
        os.makedirs('audit', exist_ok=True)
        open(AUDIT,'a').write(json.dumps({'ts': time.time(), 'x': list(x),
                                          'pred': pred})+'\n')   # audit trail
        return result

class MonitoringLayer:
    def __init__(self, reference_features): self.ref = np.asarray(reference_features)
    def drift_detected(self, log=AUDIT, min_n=100):
        if not os.path.exists(log): return False
        rows = [json.loads(l) for l in open(log)]
        if len(rows) < min_n: return False
        live = np.array([r['x'][0] for r in rows[-500:]])
        return stats.ks_2samp(self.ref, live).pvalue < 0.05   # drift signal (Module 4)

Phase 3 — Polish & Production Readiness

Phase 3 closes the loop and wraps everything in a self-service control plane: a retraining trigger consuming the drift signal and routing candidates back through the gated registry, plus the control plane that exposes the whole lifecycle as paved-road SDK calls with policy enforced and an end-to-end test proving the cycle. This is where the collection of components becomes a platform, the control plane lets a user train, register, and deploy in ML terms while monitoring, lineage, fairness gating, and security are wired in by default, and the trigger makes the system self-renewing. You will implement the trigger, the control plane with its governance defaults and an escape hatch, and a test that runs one full turn, raw data to monitored, explained, gated production model and back to retraining, confirming the platform works end to end.

Analogy🏏Cricket

🏏 Think of it like cricket: the final step in building a season-long operation is the post-match review that feeds the next selection, plus the contingency plans for when things go wrong mid-tour, so the whole cycle sustains itself and survives setbacks. Just as the review converts match observations into the next round of selection, the trigger converts drift signals into the next round of training. Just as a touring side needs plans for injuries and poor form so one failure does not derail the season, the pipeline needs error handling so one component failure does not break the loop. Just as a dry-run of the whole operation before the season confirms every handoff works, an end-to-end test confirms the whole pipeline flows. The insight is that production readiness is the loop closing on itself plus the resilience to keep running when parts fail.

python

# Phase 3: retrain trigger + self-service control plane + end-to-end test.
import logging; logging.basicConfig(level=logging.INFO); log = logging.getLogger('platform')

class ControlPlane:
    """Self-service paved road: ML-term calls; governance + wiring by default."""
    def __init__(self, data, serving_cls, monitoring, escape_hatch=False):
        self.data, self.serving_cls, self.monitoring = data, serving_cls, monitoring
        self.escape_hatch = escape_hatch
    def train_register_deploy(self, max_depth=4):
        run_id = train_and_track(self.data, max_depth)        # tracked (Phase 1)
        result = promote_if_better(run_id, self.data)         # accuracy+fairness gate
        if 'PROMOTED' not in result:
            return {'status': result}                         # gate blocked it
        # Paved road: production needs approval; serving wires monitoring+explain+defence.
        return {'status': result, 'note': 'awaiting approval for Production',
                'wired': ['lineage','monitoring','fairness_gate','explanations','api_defence']}
    def maybe_retrain(self):
        if self.monitoring.drift_detected():
            log.info('Drift detected -> retraining + re-gating')
            return self.train_register_deploy()               # closes the loop
        return {'status': 'no retrain needed'}

def test_end_to_end():
    data = DataLayer()
    # 1) Offline spine: train -> gated promotion to Staging.
    run_id = train_and_track(data)
    promo = promote_if_better(run_id, data)
    assert 'PROMOTED' in promo, promo
    # 2) Promote Staging->Production (the approval step) so serving can load it.
    import mlflow
    mv = [m for m in mlflow.MlflowClient().search_model_versions(f"name='{MODEL}'")
          if m.current_stage == 'Staging'][0]
    mlflow.MlflowClient().transition_model_version_stage(MODEL, mv.version, 'Production')
    # 3) Online: defended, explained serving by stage + an audited decision.
    serving = ServingLayer(data.reference_features())
    out = serving.predict('user1', [53.6, 131.0, 0.61, 48.0])
    assert 'in_form' in out and 'top_reasons' in out, out
    # 4) Closed loop: the control plane's retrain check runs without error.
    cp = ControlPlane(data, ServingLayer, MonitoringLayer(data.reference_features()))
    assert 'status' in cp.maybe_retrain()
    print('CAPSTONE END-TO-END OK:', promo, '| served+explained+audited | loop intact')

if __name__ == '__main__':
    test_end_to_end()

Evaluation Rubric

Lineage and reproducibility: every served model traces through the registry to a tracked run, a data version, and a fixed seed (15%).
Gated promotion: candidates pass both an accuracy comparison and a fairness gate, with the first model auto-promoting and production requiring approval (15%).
Serving correctness and safety: the API loads by stage, validates inputs, rate-limits, coarsens output, and attaches explanations to predictions (15%).
Monitoring and audit: drift and fairness are computed from logged decisions, every decision is audited, and signals gate promotion and trigger retraining (15%).
Loop closure and self-service: the retrain trigger closes the cycle, and the control plane exposes the lifecycle as paved-road, governed self-service with escape hatches (20%).
Infrastructure and CI/CD integration: the platform runs on IaC with CI/CD gating model and infrastructure changes as reproducible, reviewed assets (10%).
Architecture and integration quality: components communicate via stable artifacts, governance holds at every boundary, and the system is coherent end to end (10%).

Extension Challenges: (1) Add full safe-rollout to the control plane, a newly promoted Staging model runs in shadow on mirrored traffic, then canary, then Production, with automated rollback on degraded metrics, fully integrating Module 4. (2) Add multi-tenancy with per-project quotas and isolation so several cricket teams share the platform without interfering, and expose per-project Grafana dashboards from the monitoring layer. (3) Add LLMOps to the platform, a retrieval-augmented cricket Q&A service backed by a vector store and a served LLM, governed by the same registry, monitoring, and security paved roads, demonstrating that the platform generalises from classical models to LLMs, the full sweep of the course in one system.

Learning Objectives

Technical Requirements

Architecture & Design

Phase 1 — Core Implementation

Phase 2 — Feature Completion

Phase 3 — Polish & Production Readiness

Evaluation Rubric

Submit your capstone project