Workloads Practice — Deploy and Update an App

What You'll Build

You will deploy the CricketPulse API as a production-grade Kubernetes Deployment with 3 replicas, all probes configured, and proper resource requests and limits. You will perform a rolling update from v1 to v2 (using different nginx configurations to simulate versions), use `kubectl rollout pause` to perform a manual canary check, complete the rollout, and then simulate a bad deployment by updating to a broken image. You will use `kubectl rollout status` to detect the failure and `kubectl rollout undo` to recover. Finally, you will use HPA prerequisites to scale the Deployment manually and verify that the correct number of Pods are running at all times.

Analogy🏏Cricket

🏏 Think of it like cricket: This exercise is a complete match simulation — not just batting practice. You'll face the full sequence: opening innings (initial deployment), the change of tactics mid-game (rolling update with canary check), an unexpected top-order collapse (bad deployment), and the team's recovery (rollback). Each scenario tests a different aspect of your Kubernetes deployment skills. Just as a batsman who's only practiced in nets needs match experience to handle the unexpected pressure of a live game, you need to experience deployment, update, failure, and recovery in sequence to build the operational instincts for real production management.

Prerequisites

A running Kubernetes cluster — the kind cluster from Lesson 4 works; re-create it if needed with `kind create cluster --name cricketpulse-cluster`.
kubectl configured and pointing to the cluster — verify with `kubectl cluster-info`.
Understanding of Deployment spec structure, probe configuration, and rollout commands from Lessons 5–7.
Docker installed for image operations (optional — this exercise uses public nginx images).
~30 minutes of focused time — you'll be watching rollout status in real time.

Setup & Project Structure

Create the manifest directory structure and a ConfigMap that simulates different CricketPulse API versions by serving different responses from nginx. Using ConfigMaps with nginx lets you test multi-version deployments without building custom Docker images.

Analogy🏏Cricket

🏏 Think of it like cricket: Simulating different CricketPulse API versions with a ConfigMap and stock nginx is like rehearsing a batting-order change using the same players wearing different bib numbers, instead of recruiting brand-new specialists for every drill. Just as swapping a player's instructions on a laminated card lets the coach test how the side responds to a new plan without signing anyone, injecting different response text through a ConfigMap lets nginx serve 'v1' or 'v2' without building a custom Docker image. Just as a good academy sets out a clear practice plan — where each drill lives, which card belongs to which scenario — you lay out a tidy manifest directory before starting. The payoff: you can rehearse realistic multi-version rollout behaviour quickly and cheaply, learning the mechanics without the overhead of building and pushing real container images.

bash

mkdir -p cricketpulse-k8s/workloads
cd cricketpulse-k8s

# ConfigMap simulating CricketPulse API v1 response
cat > workloads/configmap-v1.yaml << 'EOF'
apiVersion: v1
kind: ConfigMap
metadata:
  name: cricketpulse-config
  namespace: default
data:
  index.html: |
    {"version": "1.0", "service": "CricketPulse API",
     "status": "running", "features": ["live-scores"]}
EOF

# ConfigMap for v2
cat > workloads/configmap-v2.yaml << 'EOF'
apiVersion: v1
kind: ConfigMap
metadata:
  name: cricketpulse-config-v2
  namespace: default
data:
  index.html: |
    {"version": "2.0", "service": "CricketPulse API",
     "status": "running", "features": ["live-scores", "player-stats", "ipl-2024"]}
EOF

kubectl apply -f workloads/configmap-v1.yaml
kubectl apply -f workloads/configmap-v2.yaml
echo 'ConfigMaps created'

Step 1 — Foundation

Deploy CricketPulse v1 as a 3-replica Deployment with all lifecycle features configured. This is the baseline production deployment that all subsequent update operations will modify.

Analogy🏏Cricket

🏏 Think of it like cricket: Step 1 is setting the batting lineup before the first ball. You're establishing the production baseline — 3 batsmen at the crease (3 replicas), with the team's fitness protocol in place (health probes), clear performance expectations (resource requests and limits). Everything that follows depends on this solid initial setup.

bash

cat > workloads/deployment.yaml << 'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
  name: cricketpulse-api
  namespace: default
  annotations:
    kubernetes.io/change-cause: 'initial deployment v1.0'
spec:
  replicas: 3
  progressDeadlineSeconds: 120
  revisionHistoryLimit: 5
  selector:
    matchLabels:
      app: cricketpulse
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    metadata:
      labels:
        app: cricketpulse
        version: v1
    spec:
      containers:
        - name: api-server
          image: nginx:alpine
          ports:
            - containerPort: 80
          resources:
            requests: { cpu: 50m, memory: 64Mi }
            limits:   { cpu: 200m, memory: 128Mi }
          readinessProbe:
            httpGet: { path: /, port: 80 }
            initialDelaySeconds: 5
            periodSeconds: 5
            failureThreshold: 3
          livenessProbe:
            httpGet: { path: /, port: 80 }
            periodSeconds: 30
            failureThreshold: 3
          volumeMounts:
            - name: api-content
              mountPath: /usr/share/nginx/html
      volumes:
        - name: api-content
          configMap:
            name: cricketpulse-config
EOF

kubectl apply -f workloads/deployment.yaml
kubectl rollout status deployment/cricketpulse-api

# Verify 3 replicas running on worker nodes
kubectl get pods -l app=cricketpulse -o wide
kubectl get deployment cricketpulse-api

Step 2 — Core Logic

Perform a rolling update from v1 to v2 with a manual canary check using `kubectl rollout pause`. Update the Deployment to use the v2 ConfigMap, immediately pause the rollout after one Pod updates, verify the new version, then resume.

Analogy🏏Cricket

🏏 Think of it like cricket: Step 2 is the mid-series tactical review. The captain introduces a new player (v2 configuration) for one over to evaluate their performance before committing to a longer spell. The pause is the strategic timeout — the captain reviews the scoreboard for the new player's first over before deciding whether to continue the new approach or revert to the proven lineup.

bash

# Update the Deployment to use v2 ConfigMap and new version label
kubectl annotate deployment cricketpulse-api \
  kubernetes.io/change-cause='v2.0: add player stats and IPL 2024 features'

# Patch the deployment to use v2 config
kubectl patch deployment cricketpulse-api \
  --type='json' \
  -p='[{"op": "replace", "path": "/spec/template/spec/volumes/0/configMap/name", "value": "cricketpulse-config-v2"}, {"op": "replace", "path": "/spec/template/metadata/labels/version", "value": "v2"}]'

# IMMEDIATELY pause the rollout after 1 Pod starts updating
kubectl rollout pause deployment/cricketpulse-api

# Check the current state
kubectl get pods -l app=cricketpulse
# Should show: 2x version=v1, 1x version=v2 (the canary)

# Verify the canary Pod (v2) is serving the new content
CANARY_POD=$(kubectl get pods -l app=cricketpulse,version=v2 -o jsonpath='{.items[0].metadata.name}')
kubectl port-forward pod/$CANARY_POD 9090:80 &
sleep 2
curl localhost:9090  # Should return v2 JSON with new features
kill %1

# Canary looks good — resume the rollout
kubectl rollout resume deployment/cricketpulse-api
kubectl rollout status deployment/cricketpulse-api

# Verify all Pods are now v2
kubectl get pods -l app=cricketpulse --show-labels

Step 3 — Integration & Enhancement

Simulate a bad deployment by updating to a non-existent image, observe the rollout stall, detect it via rollout status, and execute an automated rollback. This is the recovery scenario every on-call engineer must know.

Analogy🏏Cricket

🏏 Think of it like cricket: Step 3 is the batting collapse scenario — a rapid loss of wickets that requires the captain to call the team together, diagnose what went wrong, and switch back to a more conservative approach to save the innings. The bad deployment is the top-order collapse; the rollback is the captain's tactical reset. Every production engineer has this experience; the exercise ensures you've handled it in a controlled environment before it happens in production.

bash

# Simulate a bad deployment with a non-existent image
kubectl annotate deployment cricketpulse-api \
  kubernetes.io/change-cause='v3.0: BROKEN — image does not exist'
kubectl set image deployment/cricketpulse-api \
  api-server=ghcr.io/srihayavadhana/cricketpulse:v99-DOES-NOT-EXIST

# Watch the rollout stall (new Pod will be in ImagePullBackOff)
kubectl get pods -l app=cricketpulse -w &
WATCH_PID=$!
sleep 10
kill $WATCH_PID

# Try rollout status (will block/fail)
timeout 30 kubectl rollout status deployment/cricketpulse-api || echo 'Rollout is stalled!'

# Diagnose: what's wrong with the new Pod?
NEW_POD=$(kubectl get pods -l app=cricketpulse -o jsonpath='{.items[?(@.status.phase=="Pending")].metadata.name}' | head -1)
[ -z "$NEW_POD" ] && NEW_POD=$(kubectl get pods -l app=cricketpulse --sort-by=.metadata.creationTimestamp -o jsonpath='{.items[-1].metadata.name}')
kubectl describe pod $NEW_POD | tail -20
# Events will show: Failed to pull image — ImagePullBackOff

# ROLLBACK — return to last known good version
kubectl rollout undo deployment/cricketpulse-api
kubectl rollout status deployment/cricketpulse-api

# Verify we're back to v2
kubectl rollout history deployment/cricketpulse-api
kubectl get pods -l app=cricketpulse --show-labels

Step 4 — Testing & Verification

Practice manual scaling and verify the Deployment maintains desired replica counts even when Pods are manually deleted.

Analogy🏏Cricket

🏏 Think of it like cricket: Manually deleting a Pod and watching the Deployment replace it is like a captain testing his team's bench depth by pulling a fielder off mid-session and seeing the twelfth man walk on instantly, unprompted. Just as a well-run squad has a standing rule — 'the field always has eleven' — the Deployment holds a declared replica count and the controller reconciles reality back to it the moment a Pod vanishes. Just as changing the declared squad size (say to a five-a-side drill) makes the coach add or bench players to match, `kubectl scale` changes the desired count and the controller adds or removes Pods accordingly. The payoff: you see firsthand that Kubernetes is declarative — you state the number you want and the system continuously enforces it, healing deletions and honouring scale changes without manual babysitting.

bash

# Scale up
kubectl scale deployment cricketpulse-api --replicas=5
kubectl rollout status deployment/cricketpulse-api
kubectl get pods -l app=cricketpulse  # Should show 5 pods

# Delete one Pod manually — Deployment should recreate it
POD_TO_DELETE=$(kubectl get pods -l app=cricketpulse -o jsonpath='{.items[0].metadata.name}')
kubectl delete pod $POD_TO_DELETE
kubectl get pods -l app=cricketpulse -w &
WATCH_PID=$!
sleep 20
kill $WATCH_PID
# Should show: deleted pod disappears, new pod appears, total stays at 5

# Scale back down
kubectl scale deployment cricketpulse-api --replicas=3

# Final: view rollout history with all changes
kubectl rollout history deployment/cricketpulse-api
# REVISION  CHANGE-CAUSE
# 1         initial deployment v1.0
# 2         v2.0: add player stats and IPL 2024 features
# 3         v3.0: BROKEN — image does not exist
# 4         ← rollback to revision 2

# Cleanup
kubectl delete deployment cricketpulse-api
kubectl delete configmap cricketpulse-config cricketpulse-config-v2
echo 'Exercise complete!'

Warning: If `kubectl rollout pause` doesn't pause quickly enough before the rollout completes (all 3 Pods update before you run the pause command), that's fine — the rollout completed successfully. Simply proceed to Step 3 (the bad deployment test). In a real environment with more replicas and slower image pulls, pause gives you a longer window. The important skill is knowing the pause/resume/undo commands and when to use them.

Extension Challenge: Implement a proper canary deployment pipeline using two separate Deployments: `cricketpulse-api-stable` (9 replicas, v2 image) and `cricketpulse-api-canary` (1 replica, v3 image). Create a Service that selects Pods from both Deployments using only the `app: cricketpulse` label — this routes approximately 10% of traffic to the canary. Monitor both Deployments separately, and when satisfied with the canary's metrics, update `cricketpulse-api-stable` to v3 and delete `cricketpulse-api-canary`. This two-Deployment canary pattern works with standard Kubernetes Services without requiring a service mesh.

Always follow `kubectl apply` with `kubectl rollout status` to confirm the rollout completed successfully — the apply command returns before any Pods are updated.
Use `kubectl rollout pause` immediately after triggering an update for high-risk changes — this creates a native Kubernetes canary: one Pod updates, old Pods continue serving, you observe before proceeding.
A bad deployment with `maxUnavailable: 0` stalls safely — old Pods keep running; only the surge Pod (the new, broken one) cycles through CrashLoopBackOff while the service remains available.
`kubectl rollout undo` reverts to the previous ReplicaSet — it's a full rolling update in reverse, zero-downtime, and completes in seconds if the old image is already cached on the nodes.
Manual Pod deletion tests the ReplicaSet controller's self-healing: the Deployment immediately creates a replacement to maintain the desired replica count.
Rollout history provides an audit trail — but only if you annotate deployments with `kubernetes.io/change-cause` before applying; empty CHANGE-CAUSE makes incident correlation impossible.