What You'll Build
In this hands-on exercise, you will build a complete MLflow experiment tracking pipeline centred on IPL cricket player performance data. You will synthesise a dataset of batting and bowling statistics for five legendary players — Rohit Sharma, Virat Kohli, MS Dhoni, Shubman Gill, and Jasprit Bumrah — then train three scikit-learn models (RandomForest, GradientBoosting, and LinearRegression) with varying hyperparameters. Every training run will be logged to MLflow with its parameters, evaluation metrics, and serialised model artefacts. Finally, you will compare all runs in the MLflow UI and promote the best model through the MLflow Model Registry from Staging to Production, giving you a production-grade experiment management workflow you can apply to any real ML project.
Analogy🏏Cricket
🏏 Think of it like cricket: Before the IPL auction, a franchise's analytics team evaluates every player across dozens of trial matches — tracking Rohit Sharma's strike rate in power plays, Virat Kohli's average against pace, MS Dhoni's finishing rate in the death overs, Shubman Gill's consistency across pitches, and Jasprit Bumrah's economy in the middle overs. Each trial is recorded in a shared logbook so the selectors can compare and pick the best combination. MLflow is exactly that shared logbook for your ML experiments — every training run is a trial match, every metric is a scorecard entry, and the Model Registry is the final squad announcement. Keep the auction framing in mind throughout the exercise, because it fixes the discipline the steps teach: a franchise never signs a player off one good highlight reel, and you never register a model off one lucky run — you log every trial, compare them on identical conditions, and promote only with the full scorecard in front of you.
Prerequisites
Before starting this exercise you should be comfortable with Python 3.8 or later, basic scikit-learn usage (fit, predict, train_test_split), and standard pandas DataFrame operations. You do not need prior MLflow experience — this exercise introduces all MLflow concepts from scratch. Ensure you have a terminal open in a virtual environment or Conda environment where you can install packages freely. Network access is required only for the initial pip install step; everything else runs entirely on your local machine. If you are working inside a Docker container or a cloud notebook, the same steps apply — just run the pip install cell first and confirm the mlflow package imports successfully before continuing.
Analogy🏏Cricket
🏏 Think of it like cricket: the kit check before a net session. Just as a batter arriving for practice needs pads, gloves, and a bat they already know how to use — but doesn't need to have faced the new bowling machine before, because today's session is exactly where they'll learn it — this exercise expects you to arrive comfortable with Python, basic scikit-learn (fit, predict, train_test_split), and pandas, while MLflow itself is taught from scratch. Just as the coach insists on a properly prepared practice pitch — a clean, dedicated strip rather than the match square — you need a clean virtual environment or Conda environment where packages can be installed freely. And just as the only outside help needed is the equipment delivery van arriving once before practice, network access is required only for the initial pip install; after that everything runs locally, whether your 'net' is a laptop, a Docker container, or a cloud notebook. The payoff: checking your kit now means the session ahead is pure skill-building, with no stoppages for missing gear.
Setup
python
# Install all required dependencies for this exercise
# Run this cell once before executing any other code
# Option 1: pip install (recommended for local environments and Colab)
!pip install mlflow scikit-learn pandas numpy
# Option 2: If using conda
# conda install -c conda-forge mlflow scikit-learn pandas numpy
# Verify installations
import mlflow
import sklearn
import pandas as pd
import numpy as np
print(f"MLflow version : {mlflow.__version__}")
print(f"Scikit-learn : {sklearn.__version__}")
print(f"Pandas : {pd.__version__}")
print(f"NumPy : {np.__version__}")
print("\nAll dependencies installed successfully!")
# Set the experiment name — all runs will appear under this group in the MLflow UI
experiment_name = "ipl_player_performance_prediction"
mlflow.set_experiment(experiment_name)
print(f"\nMLflow experiment set to: '{experiment_name}'")Step 1: Prepare the Cricket Dataset
A solid experiment starts with well-structured data. You will create a synthetic IPL player performance dataset that captures six features per player: runs scored, batting average, strike rate, number of wickets taken, economy rate, and years of experience. The prediction target is a composite performance score between 0 and 100. You seed the random generator with the legendary jersey number 45 — MS Dhoni's number — so every student in the class produces identical data and identical splits, making it easy to compare notes. After construction, inspect the first few rows to confirm the data looks realistic before moving on to model training.
Analogy🏏Cricket
🏏 Think of it like cricket: Every IPL franchise builds a player database before finalising their strategy. Rohit Sharma's profile might read 600 runs, 45 average, 140 strike rate, 0 wickets; Jasprit Bumrah's would show 20 runs, 8 average, 80 strike rate, 27 wickets, 6.5 economy. Shubman Gill's profile captures his consistency with a high average and rising strike rate. MS Dhoni's profile highlights his exceptional finishing rate. This structured database — one row per player profile, one column per attribute — is exactly what a pandas DataFrame provides. The performance score you are predicting is like an overall player rating: a single number summarising every dimension of value. Synthetic data plays the role of a simulated trial match: the point is not that the numbers are real, but that the *shape* is right — realistic column types, plausible ranges, and a target that genuinely depends on the features — so every downstream skill you practise (logging, comparing, registering) transfers unchanged to a real dataset later. The random seed is the fixed pitch report: anyone re-running your notebook gets the identical trial conditions, which is reproducibility in miniature.
python
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
# Seed matches MS Dhoni's iconic jersey number
np.random.seed(7)
# --- Anchor rows: real-world inspired stats for five IPL legends ---
rohit_stats = ["Rohit Sharma", 623, 45.2, 139.5, 0, 0.0, 17]
virat_stats = ["Virat Kohli", 741, 52.8, 137.2, 0, 0.0, 18]
dhoni_stats = ["MS Dhoni", 416, 39.6, 135.8, 0, 0.0, 20]
gill_stats = ["Shubman Gill", 890, 47.3, 149.1, 0, 0.0, 6]
bumrah_stats = ["Jasprit Bumrah", 18, 8.4, 76.3, 27, 6.52, 10]
legend_rows = pd.DataFrame(
[rohit_stats, virat_stats, dhoni_stats, gill_stats, bumrah_stats],
columns=["player_name", "runs", "batting_average",
"strike_rate", "wickets", "economy_rate", "experience_years"]
)
# --- Synthetic dataset: 295 additional simulated player profiles ---
n_synthetic = 295
synthetic_data = {
"player_name" : [f"Player_{i}" for i in range(n_synthetic)],
"runs" : np.random.randint(50, 900, n_synthetic).astype(float),
"batting_average" : np.round(np.random.uniform(5, 60, n_synthetic), 1),
"strike_rate" : np.round(np.random.uniform(60, 200, n_synthetic), 1),
"wickets" : np.random.randint(0, 30, n_synthetic).astype(float),
"economy_rate" : np.round(np.random.uniform(5, 12, n_synthetic), 2),
"experience_years": np.random.randint(1, 22, n_synthetic).astype(float),
}
ipl_batting_data = pd.concat(
[legend_rows, pd.DataFrame(synthetic_data)],
ignore_index=True
)
# --- Derive a composite performance score (target) ---
ipl_batting_data["performance_score"] = (
ipl_batting_data["batting_average"] * 0.35
+ ipl_batting_data["strike_rate"] * 0.25
+ ipl_batting_data["runs"] * 0.001
+ ipl_batting_data["wickets"] * 1.5
- ipl_batting_data["economy_rate"] * 0.8
+ ipl_batting_data["experience_years"] * 0.4
).clip(0, 100).round(2)
print("Dataset shape:", ipl_batting_data.shape)
print("\nFirst 6 rows (includes all 5 IPL legends):")
print(ipl_batting_data.head(6).to_string(index=False))
print("\nPerformance score stats:")
print(ipl_batting_data["performance_score"].describe())
# --- Feature / target split ---
player_features = ["runs", "batting_average", "strike_rate",
"wickets", "economy_rate", "experience_years"]
batting_target = "performance_score"
X = ipl_batting_data[player_features].values
y = ipl_batting_data[batting_target].values
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=7
)
# Scale features for models sensitive to magnitude
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
print(f"\nTrain size: {X_train.shape[0]} samples")
print(f"Test size: {X_test.shape[0]} samples")Step 2: Train and Log with MLflow
Now you will train three model families — RandomForestRegressor, GradientBoostingRegressor, and LinearRegression — across a small hyperparameter grid. Each training run is wrapped in an mlflow.start_run() context manager, which automatically closes and saves the run even if an exception occurs. Inside each run you log the model type and hyperparameters with mlflow.log_param(), record MAE, RMSE, and R² on the test set with mlflow.log_metric(), and persist the trained model artefact with mlflow.sklearn.log_model(). After this step launches, open a second terminal and run mlflow ui to watch runs appear live in the browser at http://127.0.0.1:5000.
Analogy🏏Cricket
🏏 Think of it like cricket: Imagine the Mumbai Indians coaching staff running three separate net-practice sessions for their fast bowling attack. In the first session Jasprit Bumrah bowls with a new-ball setup (hyperparameter: fresh pitch). In the second he tries a slower-ball strategy (hyperparameter: worn pitch). Rohit Sharma, as captain, watches every session and records each bowler's speed, swing percentage, and wicket count on a clipboard. After all sessions are done, Rohit compares clipboards and picks the setup that produced the best economy rate. mlflow.start_run() is Rohit's clipboard — every detail of every session is captured automatically so nothing is forgotten. The detail that makes the clipboard trustworthy is that it is filled in *during* the session, not reconstructed afterwards from memory: parameters are logged the moment the session starts, metrics the moment they are measured, inside the same `with` block. If a session is abandoned mid-over (your script crashes), the clipboard still shows exactly how far it got and with what settings — a failed run with recorded parameters is evidence; a failed run with no record is just a wasted afternoon.
python
import mlflow
import mlflow.sklearn
from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
import numpy as np
# Make sure the experiment is active (safe to call again)
mlflow.set_experiment(experiment_name)
# ------------------------------------------------------------------ #
# Define the hyperparameter grid #
# ------------------------------------------------------------------ #
model_configs = [
# RandomForest variants
{
"model_type" : "RandomForest",
"model" : RandomForestRegressor(n_estimators=50, max_depth=4, random_state=7),
"params" : {"n_estimators": 50, "max_depth": 4},
},
{
"model_type" : "RandomForest",
"model" : RandomForestRegressor(n_estimators=100, max_depth=6, random_state=7),
"params" : {"n_estimators": 100, "max_depth": 6},
},
{
"model_type" : "RandomForest",
"model" : RandomForestRegressor(n_estimators=200, max_depth=None, random_state=7),
"params" : {"n_estimators": 200, "max_depth": "None"},
},
# GradientBoosting variants
{
"model_type" : "GradientBoosting",
"model" : GradientBoostingRegressor(n_estimators=50, learning_rate=0.1, random_state=7),
"params" : {"n_estimators": 50, "learning_rate": 0.1},
},
{
"model_type" : "GradientBoosting",
"model" : GradientBoostingRegressor(n_estimators=100, learning_rate=0.05, random_state=7),
"params" : {"n_estimators": 100, "learning_rate": 0.05},
},
# LinearRegression baseline
{
"model_type" : "LinearRegression",
"model" : LinearRegression(),
"params" : {"fit_intercept": True},
},
]
# ------------------------------------------------------------------ #
# Training loop — one MLflow run per model configuration #
# ------------------------------------------------------------------ #
run_results = [] # store summary for Step 3
for config in model_configs:
run_name = f"{config['model_type']}_ipl_v{len(run_results)+1}"
with mlflow.start_run(run_name=run_name) as run:
# 1. Log model type tag
mlflow.set_tag("model_family", config["model_type"])
mlflow.set_tag("dataset", "ipl_batting_data")
mlflow.set_tag("author", "virat_kohli_xi") # cricket-themed tag
# 2. Log hyperparameters
mlflow.log_param("model_type", config["model_type"])
for param_name, param_value in config["params"].items():
mlflow.log_param(param_name, param_value)
mlflow.log_param("train_size", X_train.shape[0])
mlflow.log_param("test_size", X_test.shape[0])
mlflow.log_param("features", ",".join(player_features))
# 3. Train
config["model"].fit(X_train, y_train)
rohit_runs = config["model"].predict(X_train) # train predictions
y_pred = config["model"].predict(X_test)
# 4. Compute metrics
mae = mean_absolute_error(y_test, y_pred)
rmse = np.sqrt(mean_squared_error(y_test, y_pred))
r2 = r2_score(y_test, y_pred)
train_r2 = r2_score(y_train, rohit_runs)
# 5. Log metrics
mlflow.log_metric("mae", round(mae, 4))
mlflow.log_metric("rmse", round(rmse, 4))
mlflow.log_metric("r2", round(r2, 4))
mlflow.log_metric("train_r2", round(train_r2, 4))
mlflow.log_metric("overfit_gap", round(train_r2 - r2, 4))
# 6. Log the trained model artefact
mlflow.sklearn.log_model(
sk_model = config["model"],
artifact_path = "cricket_model",
registered_model_name = None, # we register in Step 3
)
run_id = run.info.run_id
run_results.append({
"run_id" : run_id,
"run_name" : run_name,
"model_type": config["model_type"],
"mae" : mae,
"rmse" : rmse,
"r2" : r2,
})
print(f"Run '{run_name}' | R²={r2:.4f} | RMSE={rmse:.4f} | run_id={run_id[:8]}...")
print("\nAll 6 runs logged to MLflow.")
print("Launch the UI with: mlflow ui")
print("Then open: http://127.0.0.1:5000")Step 3: Compare Runs and Register Best Model
With all six runs logged, you now use the MLflow Python client to programmatically search for the best run and register its model in the MLflow Model Registry. The search_runs API accepts the experiment name and an order_by clause, returning a pandas DataFrame of all runs sorted by any metric. Once you identify the winning run by the highest R² score, you call mlflow.register_model() with the run's artifact URI to create a registered model version. You then transition that version from None to Staging and finally to Production using the MlflowClient.transition_model_version_stage method, completing a full model lifecycle handoff.
Analogy🏏Cricket
🏏 Think of it like cricket: After every IPL team's trial sessions, the selection committee — think of MS Dhoni as the chairman — reviews all scorecards and announces a tiered squad list. The top performer is named in the Playing XI (Production). The runner-up is on the bench as a super-sub (Staging). Everyone else is in the reserves. Dhoni doesn't pick by gut feel; he reads the numbers: Shubman Gill's 890-run season earns him a Production slot, while other batters sit in Staging until they prove themselves. MLflow's Model Registry gives you the same tiered promotion system — Staging is the bench, Production is the Playing XI, and the version number is the match number. Note what the committee explicitly does not do: nobody walks into the dressing room and swaps a squad member by hand. Promotion happens through the official announcement channel — the registry API — so there is a timestamped record of who was promoted, when, and on what numbers. That audit trail is the difference between 'the model in production' being a fact you can prove and a rumour you have to investigate.
python
import mlflow
from mlflow.tracking import MlflowClient
client = MlflowClient()
# ------------------------------------------------------------------ #
# 1. Search all runs in the experiment and find the best one #
# ------------------------------------------------------------------ #
runs_df = mlflow.search_runs(
experiment_names=[experiment_name],
order_by=["metrics.r2 DESC"],
)
print("=== All Experiment Runs (sorted by R²) ===")
print(
runs_df[["tags.mlflow.runName", "metrics.r2",
"metrics.rmse", "metrics.mae", "metrics.overfit_gap"]]
.rename(columns={
"tags.mlflow.runName": "run_name",
"metrics.r2" : "R²",
"metrics.rmse" : "RMSE",
"metrics.mae" : "MAE",
"metrics.overfit_gap": "overfit_gap",
})
.to_string(index=False)
)
# Best run: highest R² on the test set
best_run = runs_df.iloc[0]
best_run_id = best_run["run_id"]
best_r2 = best_run["metrics.r2"]
best_run_name = best_run["tags.mlflow.runName"]
print(f"\nBest run : '{best_run_name}'")
print(f" run_id : {best_run_id}")
print(f" R² : {best_r2:.4f}")
# ------------------------------------------------------------------ #
# 2. Register the best model in the MLflow Model Registry #
# ------------------------------------------------------------------ #
registered_model_name = "ipl_performance_predictor"
model_artifact_uri = f"runs:/{best_run_id}/cricket_model"
model_version = mlflow.register_model(
model_uri = model_artifact_uri,
name = registered_model_name,
)
print(f"\nRegistered model : {registered_model_name}")
print(f"Version : {model_version.version}")
print(f"Current stage : {model_version.current_stage}")
# ------------------------------------------------------------------ #
# 3. Transition to Staging, then to Production #
# ------------------------------------------------------------------ #
# Staging — integration testing / shadow deployment
client.transition_model_version_stage(
name = registered_model_name,
version = model_version.version,
stage = "Staging",
)
print(f"\nModel v{model_version.version} promoted to: Staging")
# Simulate validation pass — in real life you would run an A/B test here
print("Running validation checks on Staging model...")
best_model = mlflow.sklearn.load_model(
f"models:/{registered_model_name}/Staging"
)
staging_preds = best_model.predict(X_test)
staging_r2 = r2_score(y_test, staging_preds)
print(f"Staging validation R² = {staging_r2:.4f}")
assert staging_r2 > 0.8, "Model did not meet R² > 0.8 threshold for Production!"
# Production — live serving
client.transition_model_version_stage(
name = registered_model_name,
version = model_version.version,
stage = "Production",
archive_existing_versions = True, # archive any previous Production version
)
print(f"Model v{model_version.version} promoted to: Production")
# ------------------------------------------------------------------ #
# 4. Load from Production and run a sample prediction #
# ------------------------------------------------------------------ #
production_model = mlflow.sklearn.load_model(
f"models:/{registered_model_name}/Production"
)
# Predict for a new player with Virat Kohli-like stats
virat_new_stats = np.array([[680, 50.1, 142.3, 0, 0.0, 19]])
virat_new_scaled = scaler.transform(virat_new_stats)
predicted_score = production_model.predict(virat_new_scaled)[0]
print(f"\nSample prediction (Virat Kohli-style profile):")
print(f" Features : runs=680, avg=50.1, sr=142.3, wkts=0, econ=0.0, exp=19 yrs")
print(f" Predicted performance score : {predicted_score:.2f} / 100")
print(f"\nModel registered and live in Production!")Testing & Verification
Run the verification script below to confirm that your MLflow setup is correct end-to-end. The script checks four things: the experiment exists and contains at least six runs, every run recorded the three required metrics (mae, rmse, r2), the registered model named ipl_performance_predictor has at least one version in Production stage, and the Production model can load and produce a valid prediction. All four checks must print PASS. If any check prints FAIL, re-read the step instructions for that check, ensure you executed the cells in order, and re-run only the failing step before re-running this verification script.
Analogy🏏Cricket
🏏 Think of it like cricket: the verification script is the match referee's pre-game inspection, and its value is exactly that it checks the *system*, not the players. Before any international fixture, the referee walks a fixed checklist — pitch hardness, boundary rope distance, floodlight levels, sightscreen operation — because a brilliant team on a defective ground still produces an invalid match. Your script does the referee's walk over the tracking setup: can it reach the tracking store, do the expected six runs exist, does each run carry its parameters and metrics, is exactly one model version sitting in Production? Each check is boring in isolation and decisive in combination. The deeper habit this builds is the MLOps one: never certify a pipeline by eyeballing one pretty dashboard screenshot, the way a referee never certifies a ground by glancing at it from the pavilion. Write the checklist as executable code, run it after every change, and let a green result — like the referee's signed pre-match form — be the only thing that clears the ground for play. When the same script runs in CI, every future teammate inherits the inspection for free.
python
import mlflow
from mlflow.tracking import MlflowClient
from sklearn.metrics import r2_score
import numpy as np
client = MlflowClient()
passed = 0
failed = 0
def check(label, condition, detail=""):
global passed, failed
status = "PASS" if condition else "FAIL"
if condition:
passed += 1
else:
failed += 1
suffix = f" → {detail}" if detail else ""
print(f"[{status}] {label}{suffix}")
print("=" * 55)
print(" MLflow Exercise Verification — IPL Performance Model")
print("=" * 55)
# CHECK 1 — Experiment exists
try:
exp = client.get_experiment_by_name(experiment_name)
check("Experiment exists", exp is not None, experiment_name)
except Exception as e:
check("Experiment exists", False, str(e))
# CHECK 2 — At least 6 runs logged
try:
runs_df = mlflow.search_runs(experiment_names=[experiment_name])
n_runs = len(runs_df)
check("At least 6 runs logged", n_runs >= 6, f"{n_runs} run(s) found")
except Exception as e:
check("At least 6 runs logged", False, str(e))
# CHECK 3 — All runs have the three required metrics
try:
required_metrics = {"metrics.mae", "metrics.rmse", "metrics.r2"}
all_have_metrics = all(
col in runs_df.columns and runs_df[col].notna().all()
for col in required_metrics
)
check("All runs have mae/rmse/r2", all_have_metrics)
except Exception as e:
check("All runs have mae/rmse/r2", False, str(e))
# CHECK 4 — Registered model has a Production version
try:
prod_versions = client.get_latest_versions(
"ipl_performance_predictor", stages=["Production"]
)
check(
"Model in Production registry",
len(prod_versions) >= 1,
f"version={prod_versions[0].version}" if prod_versions else "none found"
)
except Exception as e:
check("Model in Production registry", False, str(e))
# CHECK 5 — Production model loads and predicts
try:
prod_model = mlflow.sklearn.load_model("models:/ipl_performance_predictor/Production")
sample_input = np.array([[400, 38.5, 130.0, 5, 7.2, 8]])
sample_input_scaled = scaler.transform(sample_input)
pred = prod_model.predict(sample_input_scaled)
valid_pred = isinstance(pred, np.ndarray) and len(pred) == 1 and 0 <= pred[0] <= 110
check("Production model predicts", valid_pred, f"score={pred[0]:.2f}")
except Exception as e:
check("Production model predicts", False, str(e))
# CHECK 6 — Best model R² above threshold
try:
best_r2_val = runs_df["metrics.r2"].max()
check("Best R² > 0.80", best_r2_val > 0.80, f"best R²={best_r2_val:.4f}")
except Exception as e:
check("Best R² > 0.80", False, str(e))
print("=" * 55)
print(f"Result: {passed} passed, {failed} failed")
if failed == 0:
print("ALL CHECKS PASSED — exercise complete!")
else:
print("Fix the FAIL items above and re-run this cell.")Warning: Never call mlflow.end_run() manually inside a with mlflow.start_run() block — it will close the run prematurely and subsequent log_metric or log_param calls within the same block will silently fail or raise an error. Always let the context manager handle run lifecycle. Additionally, if you re-run Step 2 without restarting your kernel, MLflow will create duplicate runs with the same names. This is harmless but can clutter the UI — use unique run names or set the experiment to a new name when iterating.
Pro Tip
To launch the MLflow Tracking UI and browse all your runs visually, open a terminal in the same directory where you ran the training code and execute: mlflow ui. The server starts at http://127.0.0.1:5000 by default. You can filter runs by tag (e.g. model_family=RandomForest), sort columns by any metric, and overlay learning curves side by side. For team collaboration, point MLflow at a shared backend store — for example mlflow.set_tracking_uri('http://your-mlflow-server:5000') — so every data scientist's runs land in the same registry instead of isolated local mlruns/ folders.
- Use mlflow.start_run() as a context manager so runs always close cleanly, even when exceptions occur mid-training.
- Log every hyperparameter with mlflow.log_param() and every evaluation metric with mlflow.log_metric() inside the run context to keep all experiment data reproducible.
- mlflow.sklearn.log_model() serialises the trained scikit-learn model as an MLflow artefact, making it loadable later with a single mlflow.sklearn.load_model() call.
- mlflow.search_runs() returns a pandas DataFrame you can sort and filter programmatically, eliminating the need to manually compare the UI when selecting the best model.
- The MLflow Model Registry's three stages — None, Staging, Production — mirror a real deployment pipeline: archive old Production versions automatically using archive_existing_versions=True when promoting a new winner.