What Is a Runbook and Why Does DevOps Rely On Them?
Learn what a runbook is, how it differs from a postmortem, and why DevOps teams automate runbook steps to reduce toil — interview-ready answer.
Expected Interview Answer
A runbook is a documented, step-by-step procedure for diagnosing or resolving a specific operational scenario — such as a database failover or a spiking error rate — written so that any on-call engineer, not just the original author, can execute it correctly under pressure.
Runbooks turn tribal knowledge into a shared, versioned artifact: they list the trigger condition, exact diagnostic commands, decision points, and rollback steps for one known failure mode, and they live in version control alongside the service so they are reviewed and updated like code. A good runbook is scenario-specific ("Redis memory pressure" not "general troubleshooting") and assumes the reader has minimal context, since the person executing it during an incident may be a first-time on-call engineer at 3 a.m. Mature teams progressively automate runbook steps into scripts or self-healing automation, converting the runbook from a manual checklist into a triggerable remediation, and they track which steps are still manual as a backlog of toil to eliminate. Runbooks are distinct from postmortems: a runbook is written before an incident to guide response, while a postmortem is written after to capture what happened and improve the runbook itself.
- Reduces mean time to resolution by removing guesswork under pressure
- Makes on-call sustainable for engineers unfamiliar with a specific service
- Creates a versioned, reviewable source of operational knowledge
- Provides a clear path to automate away repetitive manual steps
AI Mentor Explanation
A runbook is like a team’s laminated rain-interruption protocol card kept in the dressing room: exact steps for covering the pitch, calculating Duckworth-Lewis targets, and resuming play, so any substitute manager can execute it correctly even if the regular coach is unavailable. Without it, a stand-in official might guess at the calculation and get the target wrong under time pressure. Because the card lists the trigger (rain interruption) and the exact sequence of actions, no improvisation is needed mid-match. Teams update the card each season as rules change, just as a runbook gets revised after each incident.
Step-by-Step Explanation
Step 1
Identify the trigger scenario
Write one runbook per specific, well-defined failure mode, not a generic troubleshooting guide.
Step 2
Document exact diagnostic commands
List the precise commands or dashboards to check first, in order, with expected output.
Step 3
Define decision points and rollback
Spell out branching logic — if X, do Y — and always include a safe rollback path.
Step 4
Automate and review
Convert manual steps into scripts over time and review the runbook after every real incident.
What Interviewer Expects
- Understanding that runbooks are scenario-specific, not generic guides
- Awareness that runbooks are versioned and reviewed like code
- Ability to distinguish a runbook (before/during incident) from a postmortem (after)
- Knowledge that runbooks should progressively be automated to reduce toil
Common Mistakes
- Writing one giant generic troubleshooting document instead of per-scenario runbooks
- Letting runbooks go stale and untested after infrastructure changes
- Assuming the reader has the same context as the original author
- Never automating any runbook steps, leaving all remediation manual forever
Best Answer (HR Friendly)
“A runbook is basically a step-by-step guide for handling one specific kind of problem, written clearly enough that whoever is on call — even if they have never touched that system before — can follow it and fix the issue safely. We keep them in version control next to the code, and every time we learn something new from an incident, we update the runbook so the next person has an easier time.”
Code Example
#!/usr/bin/env bash
# runbook: redis-memory-pressure.sh
# Trigger: RedisMemoryUsage alert > 90% for 5 minutes
echo "1. Checking current memory usage..."
redis-cli info memory | grep used_memory_human
echo "2. Listing top 10 largest keys..."
redis-cli --bigkeys | head -n 20
echo "3. If eviction policy is noeviction, switch to allkeys-lru:"
redis-cli config set maxmemory-policy allkeys-lru
echo "4. Escalate to #oncall-database if memory still above 90% after step 3"Follow-up Questions
- How do you keep a runbook from going stale as infrastructure changes?
- What is the difference between a runbook and a postmortem?
- How would you decide which runbook steps to automate first?
- How do you write a runbook for an engineer with zero context on the service?
MCQ Practice
1. What best characterizes a good runbook?
Effective runbooks are narrow and scenario-specific, listing exact triggers, diagnostics, and steps for one failure mode.
2. How does a runbook differ from a postmortem?
Runbooks are proactive response guides; postmortems are retrospective analyses that often feed improvements back into runbooks.
3. What is the recommended long-term evolution of a manual runbook step?
Mature teams automate repetitive runbook steps over time, reducing toil and mean time to resolution.
Flash Cards
What is a runbook? — A step-by-step, scenario-specific procedure for diagnosing or resolving a known operational issue.
Runbook vs postmortem? — Runbook guides response before/during an incident; postmortem analyzes what happened afterward.
Where should runbooks live? — In version control alongside the service, reviewed and updated like code.
Long-term goal for manual runbook steps? — Progressively automate them into scripts or self-healing remediation.