What is Retry with Exponential Backoff?
Learn retry with exponential backoff and jitter: why it prevents thundering herds and when retries are safe to use.
Expected Interview Answer
Retry with exponential backoff is a strategy for re-attempting a failed request after progressively longer wait times between each retry, often with added randomness (jitter), so transient failures get another chance without hammering an already struggling service.
Instead of retrying immediately (which can worsen an overload) or retrying at a fixed interval (which can cause many clients to retry in lockstep, a “thundering herd”), exponential backoff doubles or otherwise multiplies the wait time after each failed attempt, up to a maximum cap and a maximum number of attempts. Jitter adds a random offset to each wait so that many clients retrying around the same time do not all collide on the exact same retry instant. Retries should only be applied to idempotent operations or operations made idempotent via an idempotency key, since retrying a non-idempotent write (like charging a card twice) can cause real damage. Retry with backoff is commonly paired with a circuit breaker so retries stop entirely once a dependency is clearly down rather than continuing to retry indefinitely.
- Gives transient failures (network blips, brief overload) a chance to succeed on retry
- Progressively longer waits reduce added load on an already struggling service
- Jitter prevents synchronized retry storms from many clients at once
- Bounded attempts and max delay prevent retries from running forever
AI Mentor Explanation
Retry with backoff is like a batter who, after being beaten by a tricky delivery, does not immediately try the exact same risky shot again but instead lets a few balls pass, waiting progressively longer before attempting it once more. Each failed attempt makes them more cautious, spacing out their next try rather than repeating it back-to-back. If they keep failing, they eventually stop attempting that shot altogether rather than risking their wicket forever. That progressively longer, capped spacing between attempts is exactly what exponential backoff does for failed requests.
Step-by-Step Explanation
Step 1
Attempt the operation
The client makes a request and the call fails due to a transient error (timeout, 503, connection reset).
Step 2
Check if the failure is retryable
Only retry idempotent operations or ones protected by an idempotency key, and only on retryable error types.
Step 3
Wait with exponential backoff and jitter
Delay before the next attempt grows exponentially (e.g. base * 2^attempt) with random jitter added, capped at a max delay.
Step 4
Stop after max attempts
After a bounded number of retries, give up and surface the error rather than retrying forever.
What Interviewer Expects
- Explains why fixed-interval retries cause thundering herd problems
- Describes exponential growth of the delay and the purpose of jitter
- Mentions idempotency as a precondition for safe retries
- Names bounded max attempts / max delay and pairing with circuit breakers
Common Mistakes
- Retrying immediately with no delay, worsening an overloaded dependency
- Retrying non-idempotent operations without an idempotency key, causing duplicate side effects
- Omitting jitter, causing synchronized retry storms across many clients
- Retrying indefinitely instead of capping attempts and total delay
Best Answer (HR Friendly)
“Retry with backoff means that if a request fails, instead of trying again right away, the system waits a little longer each time before retrying, and adds some randomness so lots of clients do not all retry at once. This gives temporary problems time to resolve without piling more load onto a service that is already struggling.”
Code Example
async function retryWithBackoff(fn, { maxAttempts = 5, baseMs = 200, maxMs = 8000 } = {}) {
let attempt = 0
while (true) {
try {
return await fn()
} catch (err) {
attempt += 1
if (attempt >= maxAttempts || !isRetryable(err)) {
throw err
}
const exponentialDelay = Math.min(maxMs, baseMs * 2 ** attempt)
const jitter = Math.random() * exponentialDelay * 0.5
const delay = exponentialDelay / 2 + jitter
await new Promise((resolve) => setTimeout(resolve, delay))
}
}
}
function isRetryable(err) {
return err.code === "ETIMEDOUT" || err.status === 503 || err.status === 429
}Follow-up Questions
- Why is jitter necessary in addition to exponential growth of the delay?
- How do idempotency keys make it safe to retry a payment API call?
- How would you decide which HTTP status codes are safe to retry versus not?
- How does retry with backoff interact with a circuit breaker on the same call?
MCQ Practice
1. Why is jitter added to exponential backoff retry delays?
Jitter randomizes retry timing across clients, avoiding synchronized retry storms (thundering herd) on the dependency.
2. Why should retries generally be limited to idempotent operations?
Retrying a non-idempotent write can duplicate effects (e.g. double-charging); idempotency or idempotency keys make retries safe.
3. What does “exponential” in exponential backoff refer to?
Each failed attempt multiplies the delay before the next retry (e.g. doubling it), bounded by a maximum delay.
Flash Cards
What is exponential backoff? — Retrying a failed request with progressively longer delays between attempts, up to a cap.
Why add jitter? — To randomize retry timing and prevent many clients from retrying in synchronized lockstep.
When is it safe to retry? — When the operation is idempotent or protected by an idempotency key.
What pairs well with retry with backoff? — A circuit breaker, so retries stop entirely once a dependency is clearly down.