Practice — deploy the IPL scorecard platform to Kubernetes

What You'll Build

In this exercise you will translate the Docker Compose IPL analytics platform from M2 Exercise into a complete set of Kubernetes manifests, deploying the FastAPI scorecard service, PostgreSQL, and Redis to a local Kubernetes cluster using kind or minikube. You will apply every M3 concept in a single integrated deployment: a dedicated namespace with ResourceQuota and LimitRange, a Deployment with rolling update strategy and readiness/liveness probes, ClusterIP Services for internal communication, a ConfigMap for application configuration, a Secret for database credentials, a PodDisruptionBudget, and topology spread constraints for zone distribution. The verification sequence confirms each Kubernetes property independently — from namespace isolation through Service DNS resolution to rolling update mechanics — using the same principle-traceability discipline established in M1 and M2.

The value of this exercise is not merely translating Compose syntax to Kubernetes YAML — it is making the conceptual mapping explicit: `mem_limit: 256m` becomes `resources.limits.memory: 256Mi`, `healthcheck: test: pg_isready` becomes a `readinessProbe: exec: pg_isready`, `depends_on: condition: service_healthy` becomes an init container, and `--profile dev` becomes a separate `development` namespace. Every Compose decision that you made for a specific reason in M2 has a precise Kubernetes equivalent that you make for the same reason, and recognising that mapping is what allows you to move fluidly between local Compose development and Kubernetes deployment without treating them as conceptually separate environments.

Analogy🏏Cricket

🏏 Think of it like cricket: This lab's staging-then-production TLS pipeline is the ICC's pre-tour practice match protocol. Before the Test series proper, the touring team plays a two-day warm-up match against a local state side — not under ICC playing conditions, not with the official match balls, and not counted in official records. The warm-up match validates that the team's batting and bowling combinations work on local pitch conditions before committing to the conditions for the official five-day Test. The Let's Encrypt staging environment is that warm-up match: it issues real certificates signed by a staging CA (not trusted by browsers, like an unofficial match result), validates the complete ACME challenge flow, and confirms that DNS, network, and IAM configurations are correct — all without consuming the production rate limit. Switching to the production issuer is committing to the official Test: the certificate is now signed by Let's Encrypt's trusted CA (officially recognised), but any misconfiguration wastes an official certificate issuance. The certificate rotation simulation is the team testing their emergency substitution protocol — specifically waiting until the 'player' (certificate) is within the renewal window, confirming the automatic replacement process fires correctly, and verifying the new 'player' is ready to take the field. This reveals why the three-phase structure of the lab matters: each phase validates a distinct property of the TLS chain, and proceeding through them in order reduces the risk that a configuration problem discovered in production was one that staging testing would have caught.

Prerequisites

A local Kubernetes cluster via kind (`kind create cluster --name ipl-cluster`) or minikube (`minikube start`) — verify with `kubectl cluster-info` before beginning.
The IPL scorecard container image pushed to a registry accessible from the cluster — for kind, load the image directly with `kind load docker-image ipl-scorecard:latest --name ipl-cluster`; for minikube, use `minikube image load`.
kubectl v1.29+ installed and configured to point to the local cluster — verify with `kubectl version --client` and `kubectl config current-context`.
Familiarity with all five M3 reading lessons: the Deployment manifest from Lesson 16, the Service types from Lesson 17, ConfigMaps and Secrets from Lesson 18, and the ResourceQuota/LimitRange pattern from Lesson 19 are all applied directly in this exercise.
The M2 Exercise `compose.yml` available for reference — you will translate each Compose service definition to its Kubernetes equivalent manifest, documenting the mapping explicitly.

Setup & Project Structure

Organise the Kubernetes manifests in a `k8s/` directory with subdirectories for each concern: `namespace/`, `storage/`, `config/`, `deployments/`, and `services/`. Apply manifests in dependency order: namespace and governance objects first, then ConfigMaps and Secrets, then StatefulSets and Deployments, then Services. This ordering matches the dependency graph of the resources and prevents race conditions where a Deployment tries to mount a ConfigMap that does not yet exist.

Analogy🏏Cricket

🏏 Think of it like cricket: setting up a tournament venue is not one job but a strict sequence of jobs, and smart boards hire a professional event crew instead of doing each task by hand. Just as the event crew handles seating, accreditation desks, and broadcast rigging as one coordinated package, Helm installs the Nginx IngressController and cert-manager with all their CRDs, ServiceAccounts, and RBAC in one release instead of dozens of hand-applied manifests. And order matters: just as the DRS cameras and replay screens must be rigged and tested before the third umpire takes his seat — an official with no working replay feed is useless and the review system collapses — cert-manager's CRDs must exist before the controller starts, or it crashes on startup before its CRD admission webhooks ever come online. The payoff: a correctly sequenced, fully provisioned venue — controller infrastructure that comes up cleanly the first time.

bash

# Project structure and cluster setup

mkdir -p k8s/{namespace,storage,config,deployments,services}

# Verify cluster is accessible
kubectl cluster-info
kubectl get nodes -o wide

# For kind: load the local image into the cluster
kind load docker-image   localhost:5001/ipl-scorecard:latest   --name ipl-cluster

# Confirm the image is available on cluster nodes
kubectl run image-test   --image=localhost:5001/ipl-scorecard:latest   --image-pull-policy=Never   --command -- echo "image available"   --restart=Never   --namespace default
kubectl wait --for=condition=complete pod/image-test --timeout=30s
kubectl delete pod image-test

echo "Cluster ready. Image accessible. Beginning manifest deployment."

# Directory structure
tree k8s/
# k8s/
# ├── namespace/
# │   ├── namespace.yaml         (Namespace + ResourceQuota + LimitRange)
# ├── config/
# │   ├── configmap.yaml         (ipl-api-config ConfigMap)
# │   └── secret.yaml            (ipl-postgres-secret, ipl-redis-secret)
# ├── storage/
# │   └── postgres-pvc.yaml      (PersistentVolumeClaim for PostgreSQL data)
# ├── deployments/
# │   ├── postgres-statefulset.yaml
# │   ├── redis-deployment.yaml
# │   └── ipl-api-deployment.yaml
# └── services/
#     ├── postgres-service.yaml  (headless for StatefulSet)
#     ├── redis-service.yaml     (ClusterIP)
#     └── ipl-api-service.yaml   (ClusterIP)

Step 1 — Foundation

Create the namespace governance objects first, then the ConfigMaps and Secrets. These are the foundation that all subsequent workloads depend on: a Deployment that references a non-existent ConfigMap will fail to start, and a namespace without a LimitRange will reject Pods that omit resource requests as soon as the ResourceQuota is applied. The governance objects change infrequently — they are the stable layer of the configuration hierarchy — and committing them to version control before any workload manifests ensures that the cluster's governance baseline is reproducible independently of any application code.

Analogy🏏Cricket

🏏 Think of it like cricket: before the first official fixture, a serious team plays a full-dress practice match — real pitch, real umpires, real match conditions — knowing the result won't appear in any league table. Just as that practice match earns no points but proves the batting order, bowling plans, and fielding drills actually work under match pressure, the Let's Encrypt staging certificate won't be trusted by any browser but proves that the ACME challenge completes, DNS resolves correctly, and the IngressController routes traffic as designed. Just as a coach would never debut an untested game plan in a televised final, you never point at the production issuer first — a failed live attempt costs far more than a failed rehearsal. The payoff: every moving part of the certificate pipeline validated cheaply, so the later switch to production is a formality rather than a gamble.

yaml

# k8s/namespace/namespace.yaml — namespace + governance bundle
apiVersion: v1
kind: Namespace
metadata:
  name: ipl-production
  labels:
    pod-security.kubernetes.io/enforce: restricted
---
apiVersion: v1
kind: ResourceQuota
metadata:
  name: ipl-production-quota
  namespace: ipl-production
spec:
  hard:
    requests.cpu: "4"
    requests.memory: 8Gi
    limits.cpu: "8"
    limits.memory: 16Gi
    count/pods: "20"
    count/services: "10"
    services.nodeports: "0"
---
apiVersion: v1
kind: LimitRange
metadata:
  name: ipl-production-limits
  namespace: ipl-production
spec:
  limits:
    - type: Container
      default: {cpu: 200m, memory: 256Mi}
      defaultRequest: {cpu: 100m, memory: 128Mi}
      max: {cpu: "1", memory: 1Gi}
      min: {cpu: 50m, memory: 64Mi}

---
# k8s/config/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: ipl-api-config
  namespace: ipl-production
data:
  LOG_LEVEL: "INFO"
  FEATURE_REDIS_STATS: "true"
  APP_HOST: "0.0.0.0"
  APP_PORT: "8000"

---
# k8s/config/secret.yaml (dev only — use ExternalSecret in production)
apiVersion: v1
kind: Secret
metadata:
  name: ipl-postgres-secret
  namespace: ipl-production
type: Opaque
stringData:     # stringData: plain text, kubectl encodes to base64 automatically
  password: "ipl_dev_password_change_in_prod"
  database_url: "postgresql://rohit_admin:ipl_dev_password_change_in_prod@ipl-postgres:5432/cricket_stats"

---
kubectl apply -f k8s/namespace/
kubectl apply -f k8s/config/
kubectl get configmap,secret -n ipl-production  # confirm both exist before proceeding

Step 2 — Core Logic

Deploy PostgreSQL as a StatefulSet (the correct Kubernetes workload for databases with stable identity and persistent storage) and Redis as a Deployment (correct for stateless replicated caches). The PostgreSQL StatefulSet uses a headless Service and a volumeClaimTemplate to provision a PersistentVolumeClaim for each replica automatically. Then deploy the IPL API Deployment with an init container that waits for PostgreSQL readiness — the Kubernetes translation of the Compose `depends_on: condition: service_healthy` pattern. Deploy all four Services and verify connectivity using `kubectl exec` into the API Pod.

Analogy🏏Cricket

🏏 Think of it like cricket: once the practice match proves the plans work, the team steps into the official fixture — and everything must now count for real. Just as the practice-game scorecard is torn up so the official scorers start a fresh book, the staging certificate Secret is deleted so cert-manager issues a fresh production certificate rather than serving the old untrusted one. Just as the umpires formally verify the match ball and playing conditions before play begins, you verify the production certificate against the system certificate store — curl without -k must succeed. Then comes rehearsing the handover: just as a club renews a key player's contract well before it expires so there is never a day without him under contract, patching renewBefore to 89 days forces cert-manager to renew the 90-day certificate almost immediately, proving rotation happens automatically long before expiry. The payoff: trusted TLS in production plus demonstrated automatic renewal — no midnight expiry emergencies.

yaml

# k8s/deployments/postgres-statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: ipl-postgres
  namespace: ipl-production
spec:
  serviceName: ipl-postgres   # headless Service name for stable DNS
  replicas: 1
  selector:
    matchLabels:
      app: ipl-postgres
  template:
    metadata:
      labels:
        app: ipl-postgres
    spec:
      containers:
        - name: postgres
          image: postgres:16-alpine
          env:
            - name: POSTGRES_DB
              value: cricket_stats
            - name: POSTGRES_USER
              value: rohit_admin
            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: ipl-postgres-secret
                  key: password
          ports:
            - containerPort: 5432
          resources:
            requests: {cpu: 200m, memory: 256Mi}
            limits: {cpu: "1", memory: 1Gi}
          readinessProbe:
            exec:
              command: ["pg_isready", "-U", "rohit_admin", "-d", "cricket_stats"]
            initialDelaySeconds: 10
            periodSeconds: 5
          securityContext:
            runAsNonRoot: true
            runAsUser: 999      # postgres official image UID
            readOnlyRootFilesystem: false   # postgres writes to /var/lib/postgresql
          volumeMounts:
            - name: postgres-data
              mountPath: /var/lib/postgresql/data
  # volumeClaimTemplate: automatically creates a PVC per replica
  volumeClaimTemplates:
    - metadata:
        name: postgres-data
      spec:
        accessModes: [ReadWriteOnce]
        resources:
          requests:
            storage: 5Gi    # adjust for actual data size

---
# k8s/services/postgres-service.yaml — headless: direct Pod DNS
apiVersion: v1
kind: Service
metadata:
  name: ipl-postgres
  namespace: ipl-production
spec:
  clusterIP: None
  selector:
    app: ipl-postgres
  ports:
    - port: 5432

---
# k8s/deployments/ipl-api-deployment.yaml (abbreviated — full manifest from Lesson 16)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ipl-api
  namespace: ipl-production
  annotations:
    kubernetes.io/change-cause: "initial deployment from M3 exercise"
spec:
  replicas: 2
  selector:
    matchLabels: {app: ipl-api, tier: backend}
  strategy:
    type: RollingUpdate
    rollingUpdate: {maxSurge: 1, maxUnavailable: 0}
  template:
    metadata:
      labels: {app: ipl-api, tier: backend}
    spec:
      initContainers:
        - name: wait-for-postgres
          image: postgres:16-alpine
          command: ["sh", "-c",
            "until pg_isready -h ipl-postgres -U rohit_admin; do sleep 2; done"]
      containers:
        - name: ipl-api
          image: localhost:5001/ipl-scorecard:latest
          imagePullPolicy: Never   # kind cluster: image loaded locally
          envFrom:
            - configMapRef:
                name: ipl-api-config
          env:
            - name: DATABASE_URL
              valueFrom:
                secretKeyRef:
                  name: ipl-postgres-secret
                  key: database_url
          ports:
            - containerPort: 8000
              name: http
          resources:
            requests: {cpu: 100m, memory: 128Mi}
            limits: {cpu: "500m", memory: 256Mi}
          readinessProbe:
            httpGet: {path: /health, port: 8000}
            initialDelaySeconds: 5
            periodSeconds: 10
          livenessProbe:
            httpGet: {path: /health, port: 8000}
            initialDelaySeconds: 30
            periodSeconds: 15
          securityContext:
            runAsNonRoot: true
            runAsUser: 10001
            readOnlyRootFilesystem: true
            allowPrivilegeEscalation: false
            capabilities: {drop: ["ALL"]}
          volumeMounts:
            - {name: tmp, mountPath: /tmp}
      volumes:
        - name: tmp
          emptyDir: {sizeLimit: "32Mi"}

Step 3 — Integration & Enhancement

Apply all manifests in dependency order, wait for all Pods to reach Ready state, then run the verification sequence that confirms each Kubernetes property independently. Add a PodDisruptionBudget to protect the API Deployment during node maintenance, and test the rolling update workflow by updating the image tag and watching the deployment progress via `kubectl rollout status`.

Analogy🏏Cricket

🏏 Think of it like cricket: on the eve of a tournament the venue runs a full walk-through — a spectator's journey from the car park, through the ticket gate, to the correct stand, with stewards redirecting anyone who wanders toward the wrong entrance. Just as that walk-through exercises every link in the chain, the end-to-end check verifies DNS resolution, the TLS handshake with the production certificate, routing to the correct backend service, and the HTTPS redirect that steers plain-HTTP stragglers to the secure entrance. Just as gate staff read the stand printed on each ticket and route spectators accordingly, host-based routing reads the hostname and sends pgAdmin traffic to its own backend while API traffic goes to the API. And just as an all-venue tournament pass admits its holder to every ground without a separate ticket per stadium, the wildcard certificate covers both subdomains with a single credential. The payoff: one entry point, correctly securing and routing every kind of visitor.

bash

# Apply manifests in dependency order and verify

# Apply in dependency order
kubectl apply -f k8s/deployments/postgres-statefulset.yaml
kubectl apply -f k8s/services/postgres-service.yaml
kubectl apply -f k8s/deployments/redis-deployment.yaml
kubectl apply -f k8s/services/redis-service.yaml
kubectl apply -f k8s/deployments/ipl-api-deployment.yaml
kubectl apply -f k8s/services/ipl-api-service.yaml

# Wait for all Pods to become Ready
kubectl wait --for=condition=ready pod   -l app=ipl-postgres -n ipl-production --timeout=120s
kubectl wait --for=condition=ready pod   -l app=ipl-api -n ipl-production --timeout=120s

# ── Verification checklist ─────────────────────────────────────────────────
echo "=== M3 Exercise Verification ==="

# 1. All Pods running and ready
echo -n "1. All Pods ready:                    "
NOT_READY=$(kubectl get pods -n ipl-production   --field-selector=status.phase!=Running   -o name 2>/dev/null | wc -l)
[ "$NOT_READY" -eq "0" ] && echo "PASS ✓" || echo "FAIL ✗ ($NOT_READY not running)"

# 2. PostgreSQL headless DNS resolves to Pod IP
echo -n "2. PostgreSQL headless DNS works:     "
kubectl exec -n ipl-production   $(kubectl get pod -l app=ipl-api -n ipl-production -o name | head -1)   -- nslookup ipl-postgres.ipl-production.svc.cluster.local   | grep -q "Address" && echo "PASS ✓" || echo "FAIL ✗"

# 3. ConfigMap values injected into API container
echo -n "3. ConfigMap injection works:         "
LOG_LEVEL=$(kubectl exec -n ipl-production   $(kubectl get pod -l app=ipl-api -n ipl-production -o name | head -1)   -- sh -c "echo \$LOG_LEVEL")
[ "$LOG_LEVEL" = "INFO" ] && echo "PASS ✓" || echo "FAIL ✗ (got: $LOG_LEVEL)"

# 4. Secret file-mounted (not as env var)
echo -n "4. Secret mounted as file:            "
kubectl exec -n ipl-production   $(kubectl get pod -l app=ipl-api -n ipl-production -o name | head -1)   -- sh -c "test -f /run/secrets/postgres/password"   && echo "PASS ✓" || echo "FAIL ✗"

# 5. ResourceQuota shows usage within limits
echo -n "5. ResourceQuota not exceeded:        "
kubectl describe resourcequota ipl-production-quota -n ipl-production   | grep -v "EXCEEDED" && echo "PASS ✓" || echo "FAIL ✗"

# 6. Rolling update: change image tag and verify
echo -n "6. Rolling update completes:          "
kubectl set image deployment/ipl-api   ipl-api=localhost:5001/ipl-scorecard:latest   -n ipl-production
kubectl rollout status deployment/ipl-api -n ipl-production --timeout=120s   && echo "PASS ✓" || echo "FAIL ✗"

echo "=== Exercise verification complete ==="

Step 4 — Testing & Verification

Port-forward the API Service to localhost and run the integration tests from M1 Exercise against the Kubernetes-deployed application to confirm that the containerised workload behaves identically whether orchestrated by Compose or by Kubernetes — validating the principle that the same image runs the same way in any OCI-compliant environment, and that the Kubernetes manifest faithfully encodes the same operational configuration as the Compose file.

Analogy🏏Cricket

🏏 Think of it like cricket: at the midpoint of a season, a good head coach doesn't just check the points table — he traces how every department produced it: the academy that developed the players, the selectors who picked the squad, and the matchday operations that got the team onto the field. Just as that review makes the whole club visible as one system rather than isolated departments, the M1–M4 architecture summary traces each component's role — the container image layer where the application is built and packaged, the Kubernetes workload layer where Deployments, StatefulSets, and autoscaling run it, and the external access layer where Ingress and TLS expose it to the world. Just as ticking the final checklist confirms match-readiness, completing the lab checklist confirms every layer actually works together. The payoff: you can explain the entire platform end to end — the mark of real understanding, and the foundation M5's security work builds on.

bash

# Port-forward and integration test

# Port-forward the API Service to localhost for testing
kubectl port-forward service/ipl-api 8080:8000   -n ipl-production &
PF_PID=$!
sleep 3  # wait for port-forward to establish

# Run API integration tests against Kubernetes-deployed application
echo "Testing Kubernetes-deployed application..."
curl -s http://localhost:8080/health | python3 -m json.tool
# Expected: {"status": "healthy", "service": "ipl-scorecard"}

curl -s http://localhost:8080/batters/rohit_sharma | python3 -m json.tool
# Expected: Rohit Sharma stats JSON

curl -s http://localhost:8080/batters | python3 -c "
import sys, json
data = json.load(sys.stdin)
assert len(data) == 3, f'Expected 3 batters, got {len(data)}'
print('All 3 batters returned ✓')
"

# Run pytest suite against the Kubernetes service (same tests as M1 Exercise)
pip install httpx pytest
BASE_URL=http://localhost:8080 pytest tests/ -v --tb=short
# Expected: 4 passed (same tests pass against K8s as against local Compose)

kill $PF_PID

echo "Kubernetes deployment verified — same tests pass as Compose ✓"
echo ""
echo "Compose-to-Kubernetes mapping summary:"
echo "  mem_limit: 256m              → resources.limits.memory: 256Mi ✓"
echo "  healthcheck: pg_isready      → readinessProbe: exec: pg_isready ✓"
echo "  depends_on: service_healthy  → initContainer: wait-for-postgres ✓"
echo "  named volume                 → StatefulSet volumeClaimTemplate ✓"
echo "  user-defined network         → Namespace + ClusterIP Services ✓"
echo "  profiles: [dev]              → separate development namespace ✓"

Warning: Never use `imagePullPolicy: Never` in production — it only works for images pre-loaded onto cluster nodes and will cause ImagePullBackOff on any node that does not have the image cached locally. This policy is appropriate only for local kind or minikube development clusters where you have loaded the image directly. In all other environments, use `imagePullPolicy: IfNotPresent` for digest-pinned images (which never change, so pulling is only needed once per node) or `imagePullPolicy: Always` if you use mutable tags (which should be avoided in production). The correct production pattern — digest-pinned image with `IfNotPresent` — combines reproducibility with minimal registry round-trips.

Extension Challenge: Translate the monitoring profile from the M2 Exercise extension into Kubernetes manifests — a Prometheus Deployment with a ConfigMap for `prometheus.yml`, a ServiceMonitor CRD (from the Prometheus Operator) that scrapes the FastAPI `/metrics` endpoint, and a Grafana Deployment backed by a ConfigMap containing the JSON dashboard definition. This exercises Kubernetes ConfigMap volume mounting with multi-key file content, the ServiceMonitor pattern for declarative scrape configuration, and how Kubernetes Deployments replace Compose's monitoring profile pattern with namespace-scoped resource organization.

Every Compose service definition maps to Kubernetes equivalents: `mem_limit` → `resources.limits`, `healthcheck` → `readinessProbe`, `depends_on: service_healthy` → `initContainer`, named volume → `volumeClaimTemplate`.
Apply manifests in dependency order: namespace governance first, then ConfigMaps and Secrets, then StatefulSets, then Deployments, then Services — each layer depends on the previous being present.
PostgreSQL belongs in a StatefulSet with a headless Service and `volumeClaimTemplate`, not a Deployment — StatefulSets provide stable Pod names, ordered operations, and per-replica PVC management that databases require.
The init container pattern is the Kubernetes translation of `depends_on: condition: service_healthy` — it enforces readiness-based dependency ordering without requiring any Kubernetes-specific application code changes.
Port-forwarding with `kubectl port-forward service/<name> <local>:<remote>` allows running the same integration test suite against the Kubernetes-deployed application as against the local Compose application, confirming environment equivalence.