Error 429: What Rate Limiting Is and How to Use It

Has it ever happened to you that you open a platform and suddenly see a feature blocked, if not the entire page, because of an HTTP error 429, “Too Many Requests”? The reason lies in rate limiting.

As a user it can certainly be annoying, but it is actually a sensible design choice; sooner or later anyone who develops APIs has to implement this kind of control. So let’s see how and why to impose a limit on your users.

What rate limiting is

Rate limiting is a rule that limits how many times a subject can perform a given operation within a given period. For example, it establishes that the user:

• Can try to log in up to 5 times per minute,
• Or, that they can create up to 100 tasks per minute,
• Or again, that the limit for calls of any kind is no more than 1000 per hour.

What the limit is made of

There are four components involved in rate limiting. On one side, the limited subject. It can be identified in different ways:

• By their IP address,
• With their user profile,
• With their tenant (for example, it will identify all the users of a company in a SaaS)
• Through the API key used. A second component is the resource or operation that is limited. In fact, a single call can be limited, as in the login example given earlier, or all the calls can be limited together, with a limit set globally regardless of the endpoints.

Closely tied to the resource are the limit that is imposed (for example, 100 calls) and the time range (within a minute, within an hour, etc.).

How the user runs into the limit

For every call, the subject and the limit to be applied are identified. At that point the system checks how much has already been consumed, and decides whether to proceed with the call (and update the consumption) or block it.

In an HTTP API, an exceeded limit corresponds to a “Too Many Requests” error message. The message communicated to the user is just as important as the enforcement of the limit, because:

• It guides the user through what is happening.
• It can also provide additional information, such as how long to wait before trying again. Moreover, leaving the user hanging, with no feedback, can lead them to perceive the error as a bug, decreasing their trust in the application and push them to retry immediately, worsening the very problem that the limit was meant to solve.

When limits are easy to reach, it is advisable to expose the limits in advance, so that the client can prevent the situation in which the limit is reached. To date there is no standard for communicating how long to wait before retrying, how much availability is left or when the limit resets.

Why plan for a rate limit before you need one

I know what you’re thinking: “My API is used by 10 people, who on earth is going to attack me?”. But rate limiting isn’t only there to prevent attacks: it can also prevent incidents if the client is badly written or if an automatic retry leads to a loop of requests. It is therefore good practice to add the limit when the system is calm, because by the time you need it it’s already too late: the service isn’t working and, in the worst case, it can be a very costly mistake.

What happens when there is no limit

The main reason to implement a limit is defence:

• Some endpoints may use paid services or APIs and abuse can literally affect your product’s economics.
• Even in the best case, there is the chance that the absence of a limit leads to an overload of the system. In concrete terms, if a login has no limits it is vulnerable to brute force, that is, a cyberattack based on trial and error, in the attempt to hit upon the correct credentials.

Or again, if an endpoint uses a service, usually a paid one, to send SMS or emails, for example to reset a password, the lack of limits can drain the budget in a few minutes. So, all things considered, better safe than sorry.

Not just defence: it’s also resource distribution

Rate limiting isn’t only a defensive practice. It is a product choice. A well-designed limit, in fact:

• Prevents a single user from consuming too many resources.
• Distributes resources fairly among users. The limit is often the boundary between different commercial plans of a product. This is also very common in language model APIs, which don’t just count requests but pair them with metrics such as requests per minute (RPM) or tokens per minute (TPM). This is the case, for example, with the OpenAI APIs.

In this case, the most relevant error is not our 429 but 402, Payment Required, which, although it is not a standard error for this scenario, is more specific.

How to set a rate limit

“100 requests per minute” sounds like a clear limit, doesn’t it? And yet it leaves some ambiguity: is it 100 requests in the last 60 seconds, or 100 between 12:00 and 12:01, or is there a reserve that regenerates over time? The answer to this question varies depending on the algorithm implemented.

Counting time: fixed window and sliding window

The most intuitive way to implement the algorithm is the one that includes a time window. This window, however, can be understood in different ways.

On the one hand we have a fixed time window, the so-called fixed window, which restarts from zero every minute. It has the advantage of being very simple to implement and to maintain, requiring only that a counter be kept, increasing with every request.

Obviously, though, it also has its disadvantages, first of all the one whereby if 100 requests are made at 12:00:59 and another 100 at 12:01:00, no limit is formally violated.

The alternative is to implement a different type of window, calculated over the last 60 seconds from the current moment. This is called a sliding window and it prevents the situation mentioned above.

There are several variants of it, including the sliding window log, which saves a timestamp for every request, with a higher memory cost but perfect precision, or the sliding window counter, which uses counters but approximates the value.

Handling spikes: token bucket and leaky bucket

Other methods, rather than focusing on counting time, are designed more to regulate the flow.

In the token bucket every user has a “bucket” of tokens, each operation consumes one and they regenerate over time, but never to the point of overflowing the bucket. So for example, a user may have 100 tokens, each request requires one and about 2 tokens are refilled every second.

Therefore, if availability is at its maximum, 10 requests go through immediately. If instead the bucket is empty, the request will be blocked; but after one second it will be possible to perform two new operations.

So there are two factors that can be controlled:

• The capacity, that is, how many requests can be made immediately.
• The refill rate, that is, how quickly more can be made.

The leaky bucket, on the other hand, smooths out the flow of requests. Operations enter a queue and are processed at a constant rate. By making traffic to the server steady, it prevents spikes of requests to the protected resource.

Different algorithms, however valid they all are, answer different needs. To sum up:

In conclusion, the choice of the rate limiting algorithm plays an important role, but it doesn’t count as much as knowing how to choose well where to apply it, with which limits and what happens when the limit is exceeded.

If you’d like advice on how to structure your backend architecture and your APIs, don’t hesitate to contact us. We’ll be ready to give you a hand and to suggest plenty of other ways to make your digital product safer and more efficient.