Rate limits and handling 429 responses

​

Budgets

​

How requests are counted

• Each API key has its own budget, so one busy integration cannot exhaust another’s, unless both call from the same network address and so share its budget.
• The widget has its own budgets, separate from API traffic. Each visitor is counted by the network address their requests come from, so visitors behind one address, such as an office network, share 20 requests a minute on one embed.
• MCP requests count against the API key’s budget and also against a separate MCP budget of the same size.
• A streaming answer costs one request. Its events are not counted individually.
• Every request costs the same, whether it lists assistants or generates an answer.

​

Handling 429

HTTP/1.1 429 Too Many Requests
Retry-After: 37
Content-Type: application/json

{"code": "rate_limit_exceeded", "message": "Too many requests. Please retry later."}

• Wait the number of seconds in Retry-After, then retry.
• Retry-After is omitted when no wait is known. Fall back to exponential backoff with jitter.
• Never retry in a tight loop; that consumes the budget you need to recover.
• Read the body’s code before you wait. A 429 with quota_exceeded is the monthly answer limit, and its Retry-After can be weeks: do not retry it.

import random
import time
import requests


def request_with_backoff(method: str, url: str, **kwargs) -> requests.Response:
    for attempt in range(6):
        response = requests.request(method, url, timeout=120, **kwargs)
        if response.status_code != 429:
            return response
        if response.json().get("code") == "quota_exceeded":
            return response  # the monthly answer limit: waiting minutes will not help
        retry_after = response.headers.get("Retry-After")
        delay = int(retry_after) if retry_after else min(60, 2**attempt) + random.random()
        time.sleep(delay)
    return response

​

Quotas