Model Deployment Basics Cheat Sheet
Explains how to serve trained models via REST APIs, containerize them with Docker, and choose between batch, real-time, and canary deployment patterns.
2 PagesIntermediateMar 2, 2026
Serving a Model with FastAPI
Wrap a trained model in a REST API endpoint.
python
from fastapi import FastAPIfrom pydantic import BaseModelimport joblibimport numpy as npapp = FastAPI()model = joblib.load("model.pkl")class PredictRequest(BaseModel): features: list[float]@app.post("/predict")def predict(req: PredictRequest): X = np.array(req.features).reshape(1, -1) pred = model.predict(X) return {"prediction": pred.tolist()}# Run with: uvicorn app:app --host 0.0.0.0 --port 8000
Containerizing the Service
Package the model API into a portable Docker image.
dockerfile
FROM python:3.11-slimWORKDIR /appCOPY requirements.txt .RUN pip install --no-cache-dir -r requirements.txtCOPY model.pkl app.py ./EXPOSE 8000CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]
Deployment Patterns
Common strategies for rolling out a model safely.
- Batch inference- Run predictions on a schedule (e.g. nightly) over a dataset and write results to storage
- Online/real-time inference- Model served behind an API endpoint for low-latency, per-request predictions
- Blue-green deployment- Run old (blue) and new (green) versions side by side, switch traffic once green is verified
- Canary release- Route a small percentage of traffic to the new model version before a full rollout
- Shadow deployment- New model runs alongside the production model on live traffic without affecting responses, for comparison
- Model registry- Central store (e.g. MLflow Model Registry) that versions models and tracks stage (staging/production)
Monitoring & Versioning
What to track once a model is live.
- Data drift- Statistical change in input feature distributions compared to training data
- Concept drift- Change in the relationship between inputs and the target over time, degrading model accuracy
- Latency/throughput monitoring- Track p50/p95/p99 response times and requests per second for the serving endpoint
- Model versioning- Tag each deployed artifact with a version/hash so predictions are reproducible and rollback-able
- A/B testing- Compare business metrics between model versions on live, randomly split traffic
Pro Tip
Always log the model version and input feature values alongside each prediction - without this, you cannot reproduce or debug a bad prediction after the fact.
Was this cheat sheet helpful?
Explore Topics
#ModelDeploymentBasics#ModelDeploymentBasicsCheatSheet#DataScience#Intermediate#ServingAModelWithFastAPI#ContainerizingTheService#DeploymentPatterns#MonitoringVersioning#MachineLearning#APIs#Docker#DevOps#CheatSheet#SkillVeris
