Token Bucket vs Leaky Bucket: What Is the Difference?
Understand the difference between token bucket and leaky bucket rate limiting, when to use each, and how each shapes traffic.
Expected Interview Answer
Token bucket allows bursts of traffic up to a stored capacity of accumulated tokens while enforcing a long-run average rate, whereas leaky bucket forces requests out at a strictly constant rate regardless of how bursty the incoming traffic is, so the key difference is burst tolerance versus rate smoothing.
In token bucket, tokens refill into a bucket at a steady rate up to a maximum capacity; each request consumes one token, and if the bucket has accumulated tokens from a quiet period, a client can legitimately burst through several requests instantly. In leaky bucket, incoming requests are placed into a queue (the bucket) and processed — leaked out — at a fixed, constant rate no matter how many arrive at once, so output is always smooth even if input is spiky; excess requests beyond queue capacity are dropped. Token bucket is the right choice when you want to permit legitimate bursts (a user quickly submitting a batch of actions after being idle), while leaky bucket is the right choice when a downstream system truly cannot tolerate any burst and needs a perfectly even output rate, such as protecting a fragile legacy dependency. Both are implemented with similar per-client counters, but the conceptual model — accumulate-and-spend versus queue-and-drain-steadily — is what drives the different traffic shape each produces.
- Token bucket permits legitimate bursts without punishing idle-then-active usage patterns
- Leaky bucket guarantees a perfectly smooth, predictable output rate for fragile downstreams
- Both are cheap to implement per-client with a simple counter and timestamp
- Choosing correctly between them avoids either over-throttling bursty legitimate traffic or under-protecting a sensitive dependency
AI Mentor Explanation
Token bucket is like a bowler who saves up unused overs when rain delays play, so once play resumes they can bowl several overs back-to-back to catch up, as long as tokens (overs) were banked. Leaky bucket instead is like a fixed over-rate rule that forces exactly one over to be completed every set number of minutes no matter what, with no catching up allowed — the pace stays perfectly constant. The token bucket rewards saved-up capacity with a burst, while leaky bucket refuses any burst and insists on steady, even delivery. That is the essential difference between the two throttling models.
Step-by-Step Explanation
Step 1
Model requests as tokens or queue items
Token bucket tracks a token count that refills over time; leaky bucket tracks a queue that drains at a fixed rate.
Step 2
Decide on burst behavior
Token bucket permits a burst up to the bucket capacity; leaky bucket smooths all input to one constant output rate.
Step 3
Handle excess demand
Token bucket rejects requests when tokens run out; leaky bucket rejects or drops requests once the queue is full.
Step 4
Pick based on downstream tolerance
Use token bucket for APIs that should allow legitimate bursts; use leaky bucket to protect a fragile fixed-capacity downstream.
What Interviewer Expects
- Clearly states the core distinction: burst-permitting vs. strictly constant output rate
- Describes the token bucket refill mechanism and the leaky bucket queue-and-drain mechanism
- Gives a concrete scenario for choosing each (bursty legitimate API traffic vs. protecting a fragile downstream)
- Does not conflate either algorithm with fixed/sliding window counters
Common Mistakes
- Saying the two algorithms are basically the same thing
- Forgetting that leaky bucket smooths output regardless of input burstiness
- Not explaining why token bucket is friendlier to legitimate bursty clients
- Confusing which one uses a refill rate versus a drain rate
Best Answer (HR Friendly)
“Token bucket lets someone save up unused capacity and then burst through several requests at once, which is friendly to normal, bursty usage. Leaky bucket instead forces everything out at one constant, steady pace no matter how the requests arrive, which is useful when the thing downstream genuinely cannot handle any burst at all. The choice comes down to whether you want to allow bursts or guarantee a perfectly even flow.”
Code Example
// Token bucket: allows bursts up to capacity
function tokenBucketAllow(bucket, capacity, refillPerSec, now) {
const elapsed = now - bucket.lastRefill
bucket.tokens = Math.min(capacity, bucket.tokens + elapsed * refillPerSec)
bucket.lastRefill = now
if (bucket.tokens < 1) return false // no tokens, reject
bucket.tokens -= 1
return true // allowed, possibly part of a burst
}
// Leaky bucket: enforces a constant drain rate, no bursts
function leakyBucketAllow(queue, maxQueueSize, drainPerSec, now) {
const leaked = Math.floor((now - queue.lastDrain) * drainPerSec)
queue.size = Math.max(0, queue.size - leaked)
queue.lastDrain = now
if (queue.size >= maxQueueSize) return false // queue full, reject
queue.size += 1
return true // queued, will be processed at the fixed drain rateFollow-up Questions
- Which algorithm would you pick to protect a legacy payment processor that can only handle a fixed, constant request rate?
- How would you tune token bucket capacity and refill rate for a public API?
- Can you combine token bucket at the edge with leaky bucket before a fragile downstream? Why might that make sense?
- How do both algorithms behave differently under a sustained traffic spike that never lets up?
MCQ Practice
1. What is the key behavioral difference between token bucket and leaky bucket?
Token bucket permits burst consumption of banked tokens, while leaky bucket smooths all traffic to one fixed output rate.
2. When is leaky bucket the better choice over token bucket?
Leaky bucket is ideal when a fragile downstream system requires a smooth, constant rate regardless of how bursty the incoming traffic is.
3. In a token bucket, what happens when a client has been idle for a while?
Idle time lets tokens refill up to the capacity cap, which is what enables a legitimate burst of requests afterward.
Flash Cards
Token bucket core idea? — Tokens refill steadily up to a cap; requests consume tokens, allowing controlled bursts.
Leaky bucket core idea? — Requests queue and drain at a strictly constant rate, smoothing all bursts.
When to prefer token bucket? — When legitimate bursty traffic should be allowed as long as the average rate stays bounded.
When to prefer leaky bucket? — When a fragile downstream needs a perfectly smooth, constant request rate with no bursts.