What it routes across
The catalogue covers hosted APIs, your own OpenAI-compatible endpoints, and a local runtime registered as one. What a deployment can actually serve depends on the credentials connected to it, so this page deliberately does not print a list: the answer is per deployment and changes whenever a credential does.GET /v1/models?available=true is the authoritative answer for any given deployment.
See the model catalogue for what is recorded about each model and how free-tier allowances are pooled.
How a request survives a bad provider
Upstream rate limits and outages are the normal case, not the exception, so failover is not a feature bolted on top — it is the request path.- Three normalised axes are combined per request: reliability learned from live traffic, speed measured from throughput and time-to-first-byte, and capability ranked by tier. Two guardrail multipliers then hold back a model close to its allowance or carrying a live rate-limit penalty.
- A retryable failure moves the request on, bounded by a hop count and a wall-clock budget so a failing deployment degrades instead of hanging.
- A credential that fails goes on cooldown; a model that fails repeatedly is benched across every credential that could serve it, and recovers automatically.
- When a large share of providers fails at once, the deployment enters a degraded state in which it stops exploring and sticks to the scored order of what remains.
Budgets, and what they are not
Each provider credential can carry an optional monthly cap, counted in requests and tokens. A capped-out credential is skipped; when every candidate is capped out, the request is refused with a429 carrying a Retry-After of the time until the window resets, so a caller is told when to come back rather than left to retry blindly.
The Gateway sets no prices and bills nobody — you pay your providers under your own agreements. It does keep a paid-equivalent reference table so the analytics estimated savings figure is realistic rather than priced as though every token were a frontier model’s, and that table is a dated snapshot, not a live feed. Keys and budgets and Models and cost are explicit about both.
Content safety and isolation
The Gateway routes model calls; it does not retrieve, ground or cite. Grounding belongs to the answer engine, and each control is documented where it is enforced:- Retrieved passages are presented to a model as content rather than as instructions, and an answer that the retrieved evidence does not support is withheld rather than invented. See Security and How answers work.
- The Gateway’s response cache, when it is enabled, hashes the organisation into its key, so a cache hit can never cross an organisation. The Gateway also has content checks of its own. See Guardrails and settings.
- MITHUNAI does not train models on your content. To answer a question, the relevant passages are sent to the model provider your deployment is configured with, under that provider’s terms.