Rate limits and handling 429 responses

​

Budgets

​

How requests are counted

• Each API key has its own budget, so one busy integration cannot exhaust another’s.
• The widget has its own budget, separate from API traffic.
• MCP requests count against the API key’s budget and also against a separate MCP budget of the same size.
• A streaming answer costs one request. Its events are not counted individually.
• Every request costs the same, whether it lists assistants or generates an answer.

​

Handling 429

HTTP/1.1 429 Too Many Requests
Retry-After: 37
Content-Type: application/json

{"code": "rate_limit_exceeded", "message": "Too many requests. Please retry later."}

• Wait the number of seconds in Retry-After, then retry.
• Retry-After is omitted when no wait is known. Fall back to exponential backoff with jitter.
• Never retry in a tight loop; that consumes the budget you need to recover.

import random
import time
import requests


def request_with_backoff(method: str, url: str, **kwargs) -> requests.Response:
    for attempt in range(6):
        response = requests.request(method, url, timeout=120, **kwargs)
        if response.status_code != 429:
            return response
        retry_after = response.headers.get("Retry-After")
        delay = int(retry_after) if retry_after else min(60, 2**attempt) + random.random()
        time.sleep(delay)
    return response

​

Quotas