How to Design an API Rate Limiter
Learn how to design a distributed API rate limiter with shared Redis counters, sliding-window algorithms, and atomic checks.
Expected Interview Answer
An API rate limiter design centers on a shared, low-latency counter store (typically Redis) that tracks per-client request counts using a sliding-window or token-bucket algorithm, checked at the edge (gateway or middleware) before a request reaches backend services, so overuse is rejected cheaply and consistently across all servers.
Because API servers are usually horizontally scaled, per-instance in-memory counters cannot enforce a global limit, so the limiter needs shared state — Redis with atomic INCR and TTL, or a dedicated counting service, checked via a Lua script or transaction to avoid race conditions between the read and the increment. The sliding-window-counter algorithm (a hybrid of fixed-window and sliding-log) is the common production choice because it approximates accurate rate limiting with just two counters per key, avoiding both the memory cost of sliding logs and the boundary-burst problem of naive fixed windows. The limiter sits at the API gateway layer, keyed by user ID, API key, or IP, and returns HTTP 429 with `Retry-After` and `X-RateLimit-*` headers when the limit is exceeded, letting well-behaved clients back off gracefully. For very high scale, limiter state can be sharded by key hash across multiple Redis nodes, and a local in-memory cache with a short TTL absorbs read-heavy limit checks to reduce Redis load.
- Shared counter store enforces a consistent global limit even with many API server instances
- Sliding-window-counter algorithm avoids the burst problem of naive fixed windows without the memory cost of full logs
- Enforcing limits at the gateway protects downstream services before they see excess load
- Returning 429 with rate-limit headers lets clients back off predictably instead of hammering retries
AI Mentor Explanation
An API rate limiter is like a shared scoreboard that every umpire across multiple grounds checks before allowing a bowler to send down another delivery, rather than each ground keeping its own private tally that could let a bowler exceed the six-ball-per-over cap by switching grounds. The scoreboard is updated atomically each time a ball is bowled, so two umpires checking at the same instant never both approve the same delivery slot. Once the over’s allowance is used, the bowler is told to wait, exactly like a client getting a “too many requests” response. That shared, atomically-checked counter across all grounds mirrors how a rate limiter must work across many API server instances.
Step-by-Step Explanation
Step 1
Choose a shared counter store
Use Redis (or similar) as centralized state so all API server instances see a consistent count per client key.
Step 2
Pick a windowing algorithm
Implement sliding-window counters (fixed-window plus weighted overlap) to balance accuracy and memory cost.
Step 3
Check atomically at the gateway
Increment and check the count in a single atomic operation (Lua script/transaction) before forwarding the request downstream.
Step 4
Reject with actionable feedback
Over-limit requests get HTTP 429 with Retry-After and X-RateLimit-* headers so clients can back off correctly.
What Interviewer Expects
- Identifies the need for shared/centralized state across horizontally scaled API servers
- Names a concrete algorithm (sliding-window counter, token bucket) and its trade-offs
- Explains atomicity requirements to avoid race conditions on the increment-and-check
- Discusses scaling the limiter itself (sharding by key, local caching) at very high request volume
Common Mistakes
- Using per-instance in-memory counters that do not enforce a global limit
- Ignoring race conditions between reading and incrementing the counter
- Only discussing the algorithm without addressing where in the request path the check happens
- Forgetting to return actionable headers (429, Retry-After) for well-behaved client backoff
Best Answer (HR Friendly)
“To design a rate limiter for an API, I would put a shared counter store like Redis in front of the backend servers so every instance checks the same source of truth for how many requests a client has made recently. I would use a sliding-window algorithm to avoid unfair bursts at time-window edges, check and update the count atomically to avoid race conditions, and reject over-limit requests with a clear “too many requests” response so clients know when to retry.”
Code Example
WINDOW_SECONDS = 60
LIMIT = 100
def is_allowed(redis_client, key, now):
current_window = int(now // WINDOW_SECONDS)
previous_window = current_window - 1
current_key = f"rl:{key}:{current_window}"
previous_key = f"rl:{key}:{previous_window}"
current_count = int(redis_client.get(current_key) or 0)
previous_count = int(redis_client.get(previous_key) or 0)
elapsed_in_current = now % WINDOW_SECONDS
weight = (WINDOW_SECONDS - elapsed_in_current) / WINDOW_SECONDS
estimated_count = previous_count * weight + current_count
if estimated_count >= LIMIT:
return False # caller returns 429 with Retry-After header
pipe = redis_client.pipeline()
pipe.incr(current_key)
pipe.expire(current_key, WINDOW_SECONDS * 2)
pipe.execute()
return TrueFollow-up Questions
- How would you shard rate-limiter state across multiple Redis nodes at very high scale?
- How does the sliding-window-counter algorithm differ from a true sliding-log and from token bucket?
- How would you apply different rate limits per API tier (free vs. paid customers)?
- How do you avoid Redis becoming a single point of failure for the rate limiter?
MCQ Practice
1. Why can per-server in-memory counters not enforce a correct rate limit in a horizontally scaled API?
A client’s requests can land on any server instance; without shared state, no single instance has the full picture needed to enforce a global limit.
2. What problem does the sliding-window-counter algorithm solve compared to a naive fixed window?
By blending a weighted portion of the previous window’s count with the current window, sliding-window counters avoid the double-burst possible at fixed window edges.
3. Why must the check-and-increment operation in a rate limiter be atomic?
Without an atomic read-and-increment, two simultaneous requests could each see a stale under-limit count and both be allowed, breaking the limit.
Flash Cards
Why use shared state (e.g. Redis) for a rate limiter? — So all horizontally scaled API server instances enforce the same global limit per client.
Sliding-window-counter algorithm? — Weights the previous fixed window’s count with the current window to approximate a sliding window cheaply.
Why must checks be atomic? — To prevent race conditions where concurrent requests both pass a stale under-limit check.
What should a 429 response include? — Retry-After and X-RateLimit-* headers so clients know when and how to retry.