How Stripe runs four different rate limiters, not just one

Stripe's engineers point to four distinct scenarios a single rate limiter doesn't cover: an individual user's traffic spike threatening service for everyone else, a misbehaving script sending far more requests than intended, lower-priority requests like analytics queries crowding out critical transaction traffic, and internal failures where the system needs to selectively drop some traffic just to keep functioning at all.

The common thread is that a limiter which only asks 'how many requests per second' treats every one of those situations identically - and for a company processing real financial transactions, a query listing someone's past charges and a request creating a new charge genuinely aren't the same kind of traffic, even though a naive rate limiter can't tell them apart.

The Request Rate Limiter is Stripe's most heavily used layer, restricting each user to a set number of requests per second. It's built on the token bucket algorithm specifically rather than a stricter fixed-window count, because a token bucket naturally tolerates short bursts - a legitimate spike, like traffic during a real-time sale event - as long as a user has tokens banked up, instead of punishing them the instant they cross a strict per-second line.

It's applied identically in test mode and live mode, so a developer sees the same rate-limiting behavior during development that they'll hit in production - catching a badly-behaved integration before it ships, not after. This is the limiter doing the most work day to day: Stripe describes it rejecting millions of requests a month, the large majority of them in test mode.

Rate alone doesn't catch everything. A user could stay comfortably under a per-second limit while still keeping dozens of expensive requests in flight simultaneously, each one chewing through CPU on a heavy endpoint. The Concurrent Requests Limiter caps how many of one user's requests can be active at the same time instead of how many arrive per second.

That change in what's being measured changes how clients end up behaving, too: hammering the API and retrying immediately stops working once concurrency itself is capped, which nudges well-built clients toward queuing work and backing off instead. This limiter triggers far less often than the request-rate layer - Stripe cites roughly 12,000 rejections a month - but credits it with fixing persistent performance problems on their most CPU-intensive endpoints that per-second limiting alone never caught.

The Fleet Usage Load Shedder works proactively rather than reacting to any single user's behavior. Traffic gets split into critical methods (creating a charge) and non-critical methods (listing past charges), and a fixed share of total fleet capacity is reserved specifically for critical traffic - Stripe's own example reserves 20% of capacity for critical requests, leaving non-critical traffic an 80% share to work within.

Once non-critical traffic exceeds its allotted share, it starts getting shed with a 503 - even if critical traffic is nowhere near overloading the system on its own. The point isn't to wait for an actual overload before protecting what matters most; it's to make sure charge creation can never be crowded out by a flood of listing requests, structurally, before that crowding ever becomes a real incident.

The Worker Utilization Load Shedder is the layer that only fires during genuine incidents. It monitors real-time capacity across the fleet, and when things degrade, it sheds traffic progressively in priority order: test-mode traffic first, then GET requests, then POSTs, and only critical methods if the system is still struggling after shedding everything less important.

The subtler engineering problem here is flapping - reacting too fast to a brief spike can cause a system to oscillate between shedding and restoring load in a way that's worse than doing nothing at all. Stripe tuned this layer to shed and recover gradually, over a period of minutes, specifically to avoid that oscillation. It's the least-triggered limiter of the four - around 100 rejections a month - because it exists purely for the worst days, not the routine ones.

Every one of these four limiters depends on infrastructure - a shared cache - that can itself fail. Stripe wrapped each limiter's logic in explicit exception handling so that if the rate limiter's own dependency breaks, requests are let through rather than blocked, backed by feature flags that can disable a misbehaving limiter immediately.

The lesson underneath that detail generalizes well beyond rate limiting: a safety system that can itself become the outage is worse than having no safety system at all. The correct default failure mode for something optional is 'get out of the way,' not 'block everything.'