1) Each request is identified by its API key (and possibly user/plan). 2) The gateway checks the usage counters for that key against configured limits (e.g. RPM, TPM, quota) using an algorithm (token bucket / sliding window). 3) If the limit is not exceeded — the request passes and counters are updated (in shared storage, e.g. Redis). 4) If the limit is exceeded — the request is rejected (HTTP 429, often with a Retry-After header) or delayed/queued.
Without limits, a single client can overload the backend, exhaust provider quotas or run up uncontrolled costs. Per-API-key rate limiting protects the system, ensures fair access and keeps costs under control.
Stores current usage per key (requests/tokens in a window), usually in shared storage (e.g. Redis) for cross-instance consistency.
Official
Decides whether a request fits within the limit: token bucket, leaky bucket or sliding window.
Official
Definition of limits per key/plan (RPM/TPM/quota) and the point where a request is allowed, rejected (429) or delayed.
Official
Strict global limits require a consistent counter, adding latency; local counters are faster but less accurate.
With a fixed window, all clients may hit simultaneously after reset, causing a load spike.
Missing clear headers (limit/remaining/Retry-After) makes it hard for clients to handle 429 correctly.
Time complexity: O(1) na zadanie. Space complexity: O(liczba aktywnych kluczy).
Maintaining accurate, consistent counters across many gateway instances at low latency is the main challenge (accuracy vs performance trade-off).
What is limited: requests (RPM/RPD), tokens (TPM), concurrency, budget.
Token bucket, leaky bucket, fixed/sliding window.
Per API key, per user, per plan/group, global.
The outcome depends on the current usage of the given key.
An allow/deny decision based on counters, without routing.
Checks are independent per request; the challenge is global counter consistency.
Limit checking is a lightweight, I/O-bound operation; it runs on any CPU. The key is a fast, shared counter store (e.g. Redis).
