Why Rate Limiting Matters
Rate limiting controls how often a client can call your API. Without it, a single misbehaving client — whether a buggy script, a DDoS attack, or just an enthusiastic user — can exhaust your server resources, degrade service for everyone, and rack up infrastructure costs.
But rate limiting isn't just a defensive measure. It's also a business tool: free tier users get 100 requests/day, paid users get 10,000, enterprise customers get unlimited. The same mechanism enforces both security and billing.
The Four Main Rate Limiting Algorithms
Fixed Window Counter
How it works: Count requests in fixed time windows (e.g., 1-minute buckets). Reset the counter at the start of each window.
Window: 12:00:00 – 12:01:00 → 95 requests → OK
Window: 12:01:00 – 12:02:00 → counter resets to 0
Implementation:
// Redis-based fixed window
async function fixedWindowRateLimit(clientId, limit, windowSeconds) {
const key = `ratelimit:${clientId}:${Math.floor(Date.now() / (windowSeconds * 1000))}`;
const count = await redis.incr(key);
if (count === 1) {
await redis.expire(key, windowSeconds);
}
return count <= limit;
}
Weakness: The boundary problem. If you allow 100 req/min, a client can send 100 at 12:00:59 and 100 more at 12:01:01 — 200 requests in 2 seconds. This "boundary burst" can overwhelm services that aren't designed for it.
Sliding Window Log
How it works: Record a timestamp for each request. To check if a new request is allowed, count timestamps within the last N seconds. Discard older timestamps.
Strength: Precise — no boundary burst problem.
Weakness: Memory-intensive for high-traffic APIs (storing every timestamp).
import redis
import time
def sliding_window_log(client_id: str, limit: int, window_seconds: int) -> bool:
r = redis.Redis()
key = f"ratelimit:{client_id}"
now = time.time()
window_start = now - window_seconds
pipe = r.pipeline()
pipe.zremrangebyscore(key, 0, window_start) # remove old entries
pipe.zadd(key, {str(now): now}) # add current request
pipe.zcard(key) # count requests in window
pipe.expire(key, window_seconds)
results = pipe.execute()
return results[2] <= limit
Sliding Window Counter
A more memory-efficient approximation of sliding window log. Combine the current fixed window count with a weighted portion of the previous window count.
current_requests = current_window_count + previous_window_count × (1 - elapsed_fraction)
If the limit is 100/min, it's 12:00:45 (75% through the window), and the previous window had 80 requests:
current_requests = 30 + 80 × (1 - 0.75) = 30 + 20 = 50
This approximation is within 0.003% of the exact rate for uniform traffic.
Token Bucket
How it works: Each client has a "bucket" with a maximum capacity. Tokens are added at a constant rate. Each request consumes one token. If the bucket is empty, the request is rejected (or queued).
Max capacity: 100 tokens
Refill rate: 10 tokens/second
A client can burst up to 100 requests (consuming all tokens), then can only sustain 10 requests/second.
class TokenBucket {
constructor(capacity, refillRate) {
this.capacity = capacity; // max tokens
this.tokens = capacity; // start full
this.refillRate = refillRate; // tokens per millisecond
this.lastRefill = Date.now();
}
consume(tokens = 1) {
this.refill();
if (this.tokens >= tokens) {
this.tokens -= tokens;
return true; // allowed
}
return false; // rejected
}
refill() {
const now = Date.now();
const elapsed = now - this.lastRefill;
this.tokens = Math.min(this.capacity, this.tokens + elapsed * this.refillRate);
this.lastRefill = now;
}
}
const bucket = new TokenBucket(100, 0.01); // 100 cap, 10 tokens/second
Strength: Handles bursts gracefully. A user who makes no requests for an hour can burst up to capacity limit.
Leaky Bucket
How it works: Requests enter a fixed-size queue. A worker processes them at a constant rate. If the queue is full, new requests are rejected.
This smooths traffic — no matter how requests arrive, they're served at a constant rate. Useful for protecting backend services from bursts.
Token Bucket vs Leaky Bucket:
- Token bucket: allows short bursts, smoothed long-term rate
- Leaky bucket: strict constant output rate, no bursts
Standard Rate Limit HTTP Headers
Use these headers so clients can adapt their behavior instead of blindly hitting 429 errors:
HTTP/1.1 200 OK
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 74
X-RateLimit-Reset: 1716912000 # Unix timestamp of window reset
Retry-After: 60 # seconds until retry (on 429)
The IETF RateLimit headers draft (RFC 6585 + draft-ietf-httpapi-ratelimit-headers) standardizes this:
RateLimit-Limit: 100
RateLimit-Remaining: 74
RateLimit-Reset: 60
Always return these headers — clients that respect them make fewer 429 calls and put less load on your infrastructure.
Implementation Examples
Node.js with express-rate-limit
const rateLimit = require('express-rate-limit');
const RedisStore = require('rate-limit-redis');
const redis = require('redis');
const client = redis.createClient({ url: process.env.REDIS_URL });
const limiter = rateLimit({
windowMs: 60 * 1000, // 1 minute
max: 100, // requests per window
standardHeaders: true, // return RateLimit-* headers
legacyHeaders: false,
store: new RedisStore({
sendCommand: (...args) => client.sendCommand(args),
}),
keyGenerator: (req) => req.user?.id || req.ip, // rate limit by user if auth'd
handler: (req, res) => {
res.status(429).json({
error: 'Too many requests',
retryAfter: Math.ceil(req.rateLimit.resetTime / 1000 - Date.now() / 1000),
});
},
});
app.use('/api/', limiter);
// Stricter limiter for auth endpoints
const authLimiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 10,
message: { error: 'Too many login attempts. Try again in 15 minutes.' },
});
app.use('/api/auth/', authLimiter);
Python with FastAPI + slowapi
from fastapi import FastAPI, Request
from slowapi import Limiter, _rate_limit_exceeded_handler
from slowapi.util import get_remote_address
from slowapi.errors import RateLimitExceeded
limiter = Limiter(key_func=get_remote_address)
app = FastAPI()
app.state.limiter = limiter
app.add_exception_handler(RateLimitExceeded, _rate_limit_exceeded_handler)
@app.get("/api/data")
@limiter.limit("100/minute")
async def get_data(request: Request):
return {"data": "..."}
@app.post("/api/login")
@limiter.limit("10/15minutes")
async def login(request: Request):
return {"token": "..."}
Redis Lua Script (Atomic Token Bucket)
For distributed systems, use a Lua script to make the bucket operation atomic:
-- KEYS[1] = bucket key
-- ARGV[1] = capacity, ARGV[2] = refill_rate (tokens/sec), ARGV[3] = now (ms), ARGV[4] = requested_tokens
local key = KEYS[1]
local capacity = tonumber(ARGV[1])
local refill_rate = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local requested = tonumber(ARGV[4])
local bucket = redis.call('HMGET', key, 'tokens', 'last_refill')
local tokens = tonumber(bucket[1]) or capacity
local last_refill = tonumber(bucket[2]) or now
local elapsed = (now - last_refill) / 1000
tokens = math.min(capacity, tokens + elapsed * refill_rate)
if tokens >= requested then
tokens = tokens - requested
redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)
redis.call('EXPIRE', key, math.ceil(capacity / refill_rate) + 1)
return 1 -- allowed
else
redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)
return 0 -- rejected
end
Rate Limiting by Tier
Real-world APIs differentiate by subscription tier:
const RATE_LIMITS = {
free: { requests: 100, window: 60 * 1000 },
pro: { requests: 1000, window: 60 * 1000 },
enterprise: { requests: 10000, window: 60 * 1000 },
};
function getTierLimiter(tier) {
const { requests, window } = RATE_LIMITS[tier] || RATE_LIMITS.free;
return rateLimit({
windowMs: window,
max: requests,
keyGenerator: (req) => `${req.user.id}:${req.user.tier}`,
});
}
app.use('/api/', (req, res, next) => {
const tier = req.user?.tier || 'free';
getTierLimiter(tier)(req, res, next);
});
Graceful Client-Side Handling
Don't just crash on 429. Implement exponential backoff:
async function fetchWithRetry(url, options = {}, maxRetries = 3) {
for (let attempt = 0; attempt <= maxRetries; attempt++) {
const res = await fetch(url, options);
if (res.status !== 429) return res;
if (attempt === maxRetries) throw new Error('Rate limit exceeded after retries');
const retryAfter = parseInt(res.headers.get('Retry-After') || '60', 10);
const delay = retryAfter * 1000 * Math.pow(2, attempt); // exponential
console.log(`Rate limited. Retrying in ${delay}ms...`);
await new Promise(resolve => setTimeout(resolve, delay));
}
}
→ Generate secure API tokens for your rate-limited APIs with the Token Generator.