What is Request Coalescing and How Does It Prevent Cache Stampedes?

Learn what request coalescing is, how single-flight merging prevents cache stampedes, and how it pairs with TTL jitter.

mediumQ168 of 224 in System Design Est. time: 5 minsLast updated: July 2026

Expected Interview Answer

Request coalescing is the technique of detecting that multiple concurrent requests are asking for the same uncached (or expired) resource and merging them into a single origin fetch whose result is shared with all the waiting callers, instead of letting every request independently hit the backend.

Without coalescing, a popular cache key expiring under high concurrent traffic triggers a “thundering herd” or “cache stampede”: dozens or thousands of simultaneous requests all miss the cache at once and hammer the origin database or service with duplicate work for the exact same data. A coalescing layer keeps an in-flight map keyed by the resource identifier; the first request to miss the cache registers itself as the in-flight fetcher and proceeds to the origin, while every subsequent request for that same key within the same window attaches itself to the same in-flight promise/future instead of issuing its own origin call. When the original fetch completes, its result is delivered to all waiters and written to cache, then the in-flight entry is cleared. This turns N identical concurrent origin calls into exactly one, which is essential for protecting hot keys (celebrity cache-key problem) and for shielding downstream services from load spikes correlated with cache expiry.

Collapses a burst of duplicate concurrent requests into a single origin call
Protects the origin database/service from cache-stampede load spikes
Reduces average and tail latency for waiters versus each issuing its own slow origin call
Complements (not replaces) TTL jitter and stale-while-revalidate as stampede defenses

AI Mentor Explanation

Request coalescing is like a groundskeeper noticing that a dozen fans have all asked separately whether the pitch report is ready and, instead of running to check it a dozen times, checking once and relaying that single answer to everyone waiting. The first fan to ask triggers the actual trip to the pitch; every fan who asks moments later while that trip is still happening is simply told to wait for the same answer. Once the report comes back, all fans get it at once instead of a dozen separate trips being made. That merging of duplicate concurrent asks into one real fetch is exactly what request coalescing does for a cache.

Step-by-Step Explanation

Step 1
First request misses cache
A request for key K finds no valid cache entry and registers itself as the in-flight fetcher for K.
Step 2
Concurrent requests attach to the in-flight fetch
Any request for K arriving while the fetch is pending is given the same in-flight promise instead of starting a new origin call.
Step 3
Origin fetch completes once
The single origin call returns, and its result is written to cache for future requests.
Step 4
Result is fanned out to all waiters
Every request that attached to the in-flight fetch (plus the original) receives the shared result, and the in-flight entry is cleared.

What Interviewer Expects

Names the cache stampede / thundering herd problem coalescing solves
Describes the in-flight map / single-flight mechanism that merges concurrent requests
Mentions complementary defenses: TTL jitter, stale-while-revalidate, locking
Recognizes coalescing works within a single node/process unless backed by a distributed lock

Common Mistakes

Confusing request coalescing with simple caching (coalescing is about concurrent misses, not just storing results)
Assuming coalescing alone eliminates all stampede risk without TTL jitter or backoff
Not considering that coalescing is per-process unless implemented with a distributed lock (e.g., Redis SETNX)
Forgetting to propagate the correct error to all waiters if the origin fetch fails

Best Answer (HR Friendly)

“Request coalescing means that if many requests come in at the exact same time asking for the same missing piece of data, the system only actually goes and fetches it once, and shares that one result with everyone who was waiting. This stops a popular cache entry expiring from causing a flood of duplicate, expensive calls to the database all at once.”

Code Example

Single-flight request coalescing (in-memory)

const inFlight = new Map() // key -> Promise

async function getWithCoalescing(key, fetchFromOrigin) {
  const cached = await cache.get(key)
  if (cached) return cached

  if (inFlight.has(key)) {
    // another request is already fetching this key; share its result
    return inFlight.get(key)
  }

  const promise = (async () => {
    try {
      const value = await fetchFromOrigin(key)
      await cache.set(key, value, { ttl: 60 })
      return value
    } finally {
      inFlight.delete(key) // clear once settled, success or failure
    }
  })()

  inFlight.set(key, promise)
  return promise
}

Follow-up Questions

How would you extend request coalescing to work across multiple server processes, not just one?
What is stale-while-revalidate and how does it complement request coalescing?
How does TTL jitter reduce the chance of a stampede in the first place?
What should happen to waiters if the coalesced origin fetch fails or times out?

MCQ Practice

1. What problem does request coalescing primarily solve?

2. In a single-flight coalescing implementation, what happens to the second concurrent request for the same missing key?

3. Which technique complements request coalescing to reduce the chance of a stampede occurring at all?

Flash Cards

1 / 4

What is Request Coalescing and How Does It Prevent Cache Stampedes?

Expected Interview Answer

AI Mentor Explanation

Step-by-Step Explanation

First request misses cache

Concurrent requests attach to the in-flight fetch

Origin fetch completes once

Result is fanned out to all waiters