What is Load Shedding in System Design?
Learn what load shedding is, how it prevents cascading failure under overload, and how it differs from rate limiting and circuit breakers.
Expected Interview Answer
Load shedding is the practice of deliberately rejecting or degrading a portion of incoming requests once a system approaches overload, so that the requests it still accepts can be served reliably instead of every request slowing down or failing together.
When traffic exceeds capacity without load shedding, queues build up, latency spikes, timeouts cascade, and the system can enter a collapse where it does strictly less useful work than if it had simply refused excess traffic outright. A load shedder monitors a signal like queue depth, CPU utilization, or in-flight request count, and once a threshold is crossed it starts rejecting requests, typically the cheapest to shed first, such as low-priority background jobs or non-authenticated traffic, while protecting critical paths like checkout or login. Rejected requests get a fast, cheap response (often HTTP 503 with a Retry-After header) rather than being queued and timing out expensively later. This trades a controlled, partial failure for clients under stress against an uncontrolled, total failure for everyone, which is why it is a standard defense alongside rate limiting and circuit breakers.
- Prevents total system collapse by protecting capacity for the requests it can actually serve
- Keeps latency for accepted requests bounded even during a traffic spike
- Lets operators prioritize critical traffic (checkout, auth) over low-priority traffic under stress
- Fails fast and cheaply instead of accumulating expensive timeouts and cascading failures
AI Mentor Explanation
Load shedding is like a captain deciding, once the outfield is clearly exhausted, to stop chasing every single ball to the boundary and instead concede a controlled few runs to conserve energy for the overs that actually decide the match. Rather than letting every fielder burn out trying to save every run, the captain sheds low-value chases first and protects effort for high-stakes moments like a run-out opportunity. This deliberate, prioritized conservation keeps the team functional for the rest of the innings instead of collapsing from exhaustion. That same controlled sacrifice of lower-priority work to protect critical capacity is exactly what load shedding does for an overloaded system.
Step-by-Step Explanation
Step 1
Monitor a load signal
Track queue depth, CPU, memory, or in-flight request count as a proxy for how close the system is to overload.
Step 2
Define shedding priorities
Classify traffic into tiers (critical vs. low-priority) so shedding drops the least important work first.
Step 3
Reject fast once threshold is crossed
Return a cheap, immediate rejection (e.g., HTTP 503 with Retry-After) instead of queuing and timing out expensively.
Step 4
Recover and resume
Once the load signal drops back under threshold, resume accepting the shed traffic tiers automatically.
What Interviewer Expects
- Explains why unbounded queuing under overload makes things worse (cascading failure)
- Describes prioritization: shedding low-priority traffic before critical traffic
- Mentions a concrete load signal used to trigger shedding (queue depth, CPU, concurrency)
- Distinguishes load shedding from rate limiting (per-client caps) and circuit breakers (dependency failure)
Common Mistakes
- Confusing load shedding with rate limiting — rate limiting is per-client, load shedding is system-wide overload protection
- Not mentioning prioritization between critical and low-priority traffic
- Suggesting the system should just queue everything and let it eventually process
- Forgetting to mention fast, cheap rejection responses instead of slow timeouts
Best Answer (HR Friendly)
“Load shedding means that when a system gets more traffic than it can handle, it deliberately turns away some of the less important requests quickly, so the requests that matter most still get served reliably. It is better to reject some low-priority traffic cleanly than to let everything, including the important stuff, slow down or fail together.”
Code Example
function shouldShed(request, systemLoad) {
const THRESHOLDS = {
critical: 0.98, // shed critical traffic only near total saturation
standard: 0.85,
lowPriority: 0.65, // shed low-priority traffic earliest
}
const tier = classifyPriority(request) // 'critical' | 'standard' | 'lowPriority'
return systemLoad >= THRESHOLDS[tier]
}
async function handleRequest(req, res) {
const systemLoad = getCurrentLoadRatio() // e.g., inFlightRequests / maxCapacity
if (shouldShed(req, systemLoad)) {
res.setHeader('Retry-After', '2')
return res.status(503).json({ error: 'Service under load, please retry shortly' })
}
return processRequest(req, res)
}Follow-up Questions
- How is load shedding different from rate limiting and circuit breakers?
- What signal would you use to trigger shedding: CPU, queue depth, or latency, and why?
- How would you design traffic tiers so critical requests are shed last?
- What happens to a shed request from the client’s perspective, and how should the client react?
MCQ Practice
1. What is the primary goal of load shedding?
Load shedding protects overall system health by deliberately rejecting excess load rather than letting everything degrade together.
2. Why is unbounded queuing under overload often worse than load shedding?
Without shedding, requests pile up, consume resources while waiting, and often time out anyway — wasting capacity instead of protecting it.
3. How does load shedding typically prioritize which requests to reject first?
Effective load shedding classifies traffic by priority and sheds the least critical tier first to protect essential functionality.
Flash Cards
What is load shedding? — Deliberately rejecting some requests under overload so accepted requests remain reliably served.
Why not just queue everything? — Unbounded queues cause expensive timeouts and cascading failure, wasting capacity instead of protecting it.
How does shedding prioritize traffic? — It sheds low-priority/non-critical traffic first, protecting critical paths like checkout or auth.
Typical shed response? — A fast, cheap rejection like HTTP 503 with a Retry-After header instead of a slow timeout.