What is Request Coalescing and How Does It Prevent Cache Stampedes?
Learn what request coalescing is, how single-flight merging prevents cache stampedes, and how it pairs with TTL jitter.
Expected Interview Answer
Request coalescing is the technique of detecting that multiple concurrent requests are asking for the same uncached (or expired) resource and merging them into a single origin fetch whose result is shared with all the waiting callers, instead of letting every request independently hit the backend.
Without coalescing, a popular cache key expiring under high concurrent traffic triggers a “thundering herd” or “cache stampede”: dozens or thousands of simultaneous requests all miss the cache at once and hammer the origin database or service with duplicate work for the exact same data. A coalescing layer keeps an in-flight map keyed by the resource identifier; the first request to miss the cache registers itself as the in-flight fetcher and proceeds to the origin, while every subsequent request for that same key within the same window attaches itself to the same in-flight promise/future instead of issuing its own origin call. When the original fetch completes, its result is delivered to all waiters and written to cache, then the in-flight entry is cleared. This turns N identical concurrent origin calls into exactly one, which is essential for protecting hot keys (celebrity cache-key problem) and for shielding downstream services from load spikes correlated with cache expiry.
- Collapses a burst of duplicate concurrent requests into a single origin call
- Protects the origin database/service from cache-stampede load spikes
- Reduces average and tail latency for waiters versus each issuing its own slow origin call
- Complements (not replaces) TTL jitter and stale-while-revalidate as stampede defenses
AI Mentor Explanation
Request coalescing is like a groundskeeper noticing that a dozen fans have all asked separately whether the pitch report is ready and, instead of running to check it a dozen times, checking once and relaying that single answer to everyone waiting. The first fan to ask triggers the actual trip to the pitch; every fan who asks moments later while that trip is still happening is simply told to wait for the same answer. Once the report comes back, all fans get it at once instead of a dozen separate trips being made. That merging of duplicate concurrent asks into one real fetch is exactly what request coalescing does for a cache.
Step-by-Step Explanation
Step 1
First request misses cache
A request for key K finds no valid cache entry and registers itself as the in-flight fetcher for K.
Step 2
Concurrent requests attach to the in-flight fetch
Any request for K arriving while the fetch is pending is given the same in-flight promise instead of starting a new origin call.
Step 3
Origin fetch completes once
The single origin call returns, and its result is written to cache for future requests.
Step 4
Result is fanned out to all waiters
Every request that attached to the in-flight fetch (plus the original) receives the shared result, and the in-flight entry is cleared.
What Interviewer Expects
- Names the cache stampede / thundering herd problem coalescing solves
- Describes the in-flight map / single-flight mechanism that merges concurrent requests
- Mentions complementary defenses: TTL jitter, stale-while-revalidate, locking
- Recognizes coalescing works within a single node/process unless backed by a distributed lock
Common Mistakes
- Confusing request coalescing with simple caching (coalescing is about concurrent misses, not just storing results)
- Assuming coalescing alone eliminates all stampede risk without TTL jitter or backoff
- Not considering that coalescing is per-process unless implemented with a distributed lock (e.g., Redis SETNX)
- Forgetting to propagate the correct error to all waiters if the origin fetch fails
Best Answer (HR Friendly)
“Request coalescing means that if many requests come in at the exact same time asking for the same missing piece of data, the system only actually goes and fetches it once, and shares that one result with everyone who was waiting. This stops a popular cache entry expiring from causing a flood of duplicate, expensive calls to the database all at once.”
Code Example
const inFlight = new Map() // key -> Promise
async function getWithCoalescing(key, fetchFromOrigin) {
const cached = await cache.get(key)
if (cached) return cached
if (inFlight.has(key)) {
// another request is already fetching this key; share its result
return inFlight.get(key)
}
const promise = (async () => {
try {
const value = await fetchFromOrigin(key)
await cache.set(key, value, { ttl: 60 })
return value
} finally {
inFlight.delete(key) // clear once settled, success or failure
}
})()
inFlight.set(key, promise)
return promise
}Follow-up Questions
- How would you extend request coalescing to work across multiple server processes, not just one?
- What is stale-while-revalidate and how does it complement request coalescing?
- How does TTL jitter reduce the chance of a stampede in the first place?
- What should happen to waiters if the coalesced origin fetch fails or times out?
MCQ Practice
1. What problem does request coalescing primarily solve?
Coalescing merges concurrent duplicate misses into a single origin call, preventing a thundering herd on the origin.
2. In a single-flight coalescing implementation, what happens to the second concurrent request for the same missing key?
Subsequent requests for the same key share the already-in-flight fetch rather than triggering their own origin call.
3. Which technique complements request coalescing to reduce the chance of a stampede occurring at all?
Randomizing TTLs (jitter) spreads out expirations so many keys are less likely to expire at exactly the same moment.
Flash Cards
What is request coalescing? — Merging concurrent requests for the same missing cache key into one origin fetch shared by all waiters.
What problem does it prevent? — Cache stampedes / thundering herd, where a popular key expiring causes a flood of duplicate origin calls.
What data structure typically implements it? — An in-flight map keyed by resource ID holding the pending promise/future for that key.
What complements coalescing? — TTL jitter and stale-while-revalidate, which reduce the odds of a stampede occurring at all.