Which rate-limiting algorithm should I use?

Token bucket is the best default for most APIs: it allows short bursts up to the bucket size while enforcing a long-run average rate, which matches how real clients behave. Use a sliding window when you need an accurate, abuse-resistant count over a rolling period (for example login attempts). Reach for leaky bucket when you must smooth traffic into a downstream system at a strictly constant rate. Fixed window is the simplest and cheapest but allows roughly double the limit across a window boundary.

Token bucket vs leaky bucket — what's the difference?

Both use a bucket of capacity N, but they meter different things. Token bucket adds tokens over time and spends one per request, so it permits bursts as long as tokens are saved up — it limits the average rate. Leaky bucket queues requests and drains them at a fixed rate, so output is perfectly smooth and bursts overflow and get rejected — it limits the output rate. Token bucket favors throughput and burst tolerance; leaky bucket favors a steady, predictable downstream load.

How do I rate-limit per user instead of globally?

Key the limiter by identity: use the user ID for authenticated requests, the API key for machine clients, or the client IP as a fallback for anonymous traffic. Store each key's counter or bucket in a shared fast store such as Redis so the limit holds across all your server instances, and set a TTL so idle keys expire. Combine layers in practice — a generous per-IP limit to blunt abuse plus a tighter per-user or per-endpoint limit for fairness.

What is an HTTP 429 status code?

429 Too Many Requests is the standard HTTP response when a client has sent more requests than the rate limit allows. A well-behaved API returns it with a Retry-After header telling the client how many seconds to wait, and often RateLimit headers describing the limit, remaining quota, and reset time. Clients should back off and retry rather than hammering the endpoint.

Rate Limiting Algorithms — Beyond Code & Karma

Chapter 01 — The Foundation

A flood must become a stream.
The 429 is the gate.

Rate limiting caps how many requests a client may make in a given time, so one caller can't exhaust your capacity, run up your bill, or brute-force a login. Over the limit, the server replies 429 Too Many Requests instead of doing the work. The interesting part is how you count — four algorithms accept and reject the same traffic differently.

The Secret: they all bound a rate, but they bound different rates. Token bucket bounds the average and tolerates bursts; leaky bucket bounds the output and smooths everything; the windows bound a count per period — and differ on what happens at the boundary.

Token Bucket

Tokens refill over time; each request spends one. Saved tokens let a quiet client burst. Bounds the average rate. The API default.

Leaky Bucket

Requests queue and drain at a fixed rate. Overflow is rejected. Output is perfectly smooth — ideal for a fragile downstream.

Fixed Window

One counter per window, reset at each tick. Cheapest — but you can push ~double the limit across a boundary.

Sliding Window

Counts over a rolling period; old timestamps age out continuously. No boundary to game — accurate, at more memory cost.

The Cosmic Connection

Sanyama — the discipline of restraint

Hindu Philosophy Parallel

In the Yoga Sutras, Sanyama is mastery through restraint — measured action that channels raw force toward a purpose instead of letting it run wild. A rate limiter is karma yoga in code: a sacred river channeled so it nourishes the fields instead of flooding them.

Token bucket — saved merit

Punya accumulated quietly, then spent one measured action at a time.

Leaky bucket — the steady river

A flood channeled into a constant stream that the fields can absorb.

429 — dharma's gate

Refusing excess so the system endures. The "no" that protects the whole.

"Measured action channels a flood into a steady stream, so the system endures instead of drowning."

Chapter 02 — Show Me The Code

The token bucket, in 18 lines

The default limiter, and the cleanest to read. Refill lazily on each check — no background timer needed.

token-bucket.js

// Token bucket — the default for most public APIs
class TokenBucket {
  constructor(capacity, refillPerSec) {
    this.cap = capacity;
    this.rate = refillPerSec;
    this.tokens = capacity;
    this.last = now();
  }

  allow() {
    const t = now();
    // refill based on elapsed time, capped at capacity
    this.tokens = Math.min(this.cap, this.tokens + (t - this.last) * this.rate);
    this.last = t;
    if (this.tokens >= 1) { this.tokens -= 1; return true; }  // → 200
    return false;                                       // → 429
  }
}

The lazy-refill trick: you don't need a timer adding tokens every tick. On each allow(), add (elapsed × rate) tokens, capped at capacity. State is one number and a timestamp — trivial to store per-user in Redis.

How each decides 200 vs 429

Token bucket

Token available? Spend it → 200. None left → 429. Bursts allowed up to the saved token count.

Leaky bucket

Room in the queue? Enqueue → 200. Full → overflow → 429. The queue drains at a constant rate.

Fixed window

Counter below limit this window? Increment → 200. At limit → 429. Resets to zero at the boundary.

Sliding window

Fewer than limit timestamps in the last window seconds? Record → 200. Otherwise → 429. Old stamps age out.

Chapter 03 — Live Visualization

Spend a token, hit the limit

A token bucket, capacity 10, refilling 2/sec. Send single requests or burst — watch tokens drain faster than they refill and the 429s start.

Token Bucket · cap 10 · +2/s

10 tokens

 200 0 429 0 

Chapter 04 — Deep Dive

The nuances that senior devs know

Where the basics stop and the production reality begins.

🪙 Token bucket vs leaky bucket — they meter different things

Both hold capacity N, but token bucket limits the average rate and tolerates bursts (saved tokens spend all at once), while leaky bucket limits the output rate and smooths everything (a queue draining at a fixed rate). Token bucket favors throughput; leaky bucket favors a steady, predictable downstream load. Rule of thumb: public API → token; feeding a fragile worker → leaky.

🪟 The fixed-window boundary burst

Fixed window keeps one counter per window (say 10 per 5s) and resets at the boundary. Burst right before the reset and again right after, and you push nearly double the limit in a short span — each window counts independently. In the visualizer, switch to Fixed Window and try it.

The cheap fix: a sliding-window counter approximates the true sliding log with just two numbers — a weighted blend of the current and previous window's counts. Most of the accuracy, almost none of the memory.

🔑 Per-user limiting — key it and share it

Global limits are rare. Key the limiter by identity — user ID for authenticated requests, API key for machines, client IP as the anonymous fallback. Store each key's state in a shared fast store (Redis) with a TTL so idle keys expire, so the limit holds across every server instance. In practice you layer them: a generous per-IP limit to blunt abuse plus a tighter per-user limit for fairness.

📨 Be a good citizen — 429, Retry-After, RateLimit headers

A well-behaved API doesn't just slam the door. Return 429 Too Many Requests with a Retry-After header (seconds to wait) and the RateLimit-Limit / RateLimit-Remaining / RateLimit-Reset headers so clients can self-throttle instead of hammering you. Clients should back off exponentially, not retry immediately.

Chapter 05 — Quick Reference

The four algorithms — complete cheat sheet

Algorithm	Bounds	Bursts?	Cost	Best for
Token Bucket	Average rate	Yes (up to bucket)	Cheap	Public / REST APIs
Leaky Bucket	Output rate	No	Cheap	Feeding fragile downstream
Fixed Window	Count / window	~2× at boundary	Cheapest	Coarse limits, one Redis counter
Sliding Window	Count / rolling	No boundary game	More memory	Login / OTP throttling

Chapter 06 — Dharma & The Limit

The Karma of the 429

The Karma Connection

A refusal is not always denial — sometimes it is protection. The 429 is dharma's gate: it says "not now" so the system can keep its promise to everyone else. To rate-limit is to practise aparigraha — non-grasping — refusing to let one caller consume what belongs to the whole.

The token bucket is saved merit, spent one measured action at a time. The leaky bucket is a flood made gentle. Neither hoards, neither floods — each holds the middle path between starving the system and drowning it. That balance is the discipline.

"Channel the river; do not dam it, do not let it flood." — Sanyama, interpreted for backend engineers 🙏

Chapter 07 — Test Yourself

Rate Limiting Trivia

Three questions. No cheating. Your karma is watching.

Question 1

Which algorithm lets a client who was idle suddenly send a burst, yet still bounds the long-run average rate?

Correct! 🎉 Token bucket accumulates unused tokens up to its capacity, so a quiet client can spend them all at once — a burst — while the refill rate still caps the average. That burst tolerance is exactly why it's the default for public APIs.

Question 2

A client sends 10 requests just before a fixed window resets, then 10 just after. With a limit of 10 per window, how many succeed?

Correct! 🧠 This is the classic fixed-window boundary burst: each window has its own counter, so ~2× the limit slips through in a short span around the reset. Sliding window fixes it by counting over a rolling period with no boundary to game.

Question 3 — 🌶️ Spicy

You must feed a fragile downstream service at a strictly constant rate, dropping anything it can't handle. Which limiter fits best?

Correct! 🔥 Leaky bucket drains its queue at a fixed rate, so the downstream sees a perfectly smooth load no matter how spiky the input; anything that would overflow is rejected. Token bucket would let bursts through — the opposite of what a fragile downstream wants.

The Enlightenment

The Six Truths of Rate Limiting

Token = Average

Token bucket bounds the average rate and forgives bursts up to the saved token count. The API default.

Leaky = Output

Leaky bucket bounds the output rate — a constant drain that smooths any spike, dropping overflow.

Fixed is Gameable

One counter per window is cheapest but leaks ~2× the limit across a boundary. Know the weakness.

Sliding is Honest

A rolling count has no boundary to exploit — accurate, at the price of more bookkeeping.

Key by Identity

Real limits are per-user / per-key / per-IP, shared in Redis with a TTL so they hold across instances.

429 with Grace

Return Retry-After and RateLimit headers. A good "no" tells the client exactly when to come back.

कर्म करो, फल की चिंता मत करो

Rate Limiting

A flood must become a stream.The 429 is the gate.