> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mithunai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Supported Models, LLM Providers & Inference Engines

> Configure frontier and open-weights reasoning models across Anthropic, OpenAI, Google Gemini, DeepSeek, Groq, or private on-premises vLLM servers.

MITHUNAI integrates with both frontier commercial model providers and sovereign private inference runtimes through an abstracted model catalog. Each assistant connects to a specific model identifier, routing requests through the intelligent inference gateway for semantic caching, rate limiting, and prompt injection defense.

```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart TD
    GW["MITHUNAI Inference Gateway"]

    subgraph Frontier["Cloud Model Providers"]
        CL["Anthropic Claude 3.5 Sonnet / Haiku"]
        OA["OpenAI GPT-4o / GPT-4o-mini / o1"]
        GM["Google Gemini 2.0 Flash / Pro"]
        DS["DeepSeek R1 / V3"]
        GQ["Groq Llama 3.3 70B (Fast Inference)"]
    end

    subgraph Sovereign["Private On-Premises Runtimes"]
        VLLM["vLLM Cluster\n(High-Throughput PagedAttention)"]
        OL["Ollama Server\n(Local Developer Inference)"]
        TRT["NVIDIA Triton Inference Server"]
    end

    GW --> Frontier
    GW --> Sovereign
```

***

## Supported Frontier Models

Frontier models deliver high reasoning precision for complex technical inquiries, code synthesis, and multi-document synthesis:

| Provider      | Model Identifier              | Recommended Use Case                                                            | Context Window   |
| :------------ | :---------------------------- | :------------------------------------------------------------------------------ | :--------------- |
| **Anthropic** | `anthropic/claude-3-5-sonnet` | Complex technical reasoning, architecture Q\&A, and nuanced code generation.    | 200,000 tokens   |
| **Anthropic** | `anthropic/claude-3-5-haiku`  | High-volume, low-latency customer support and quick factual lookups.            | 200,000 tokens   |
| **OpenAI**    | `openai/gpt-4o`               | Multimodal comprehension, structured JSON output, and general engineering docs. | 128,000 tokens   |
| **OpenAI**    | `openai/gpt-4o-mini`          | Cost-effective tier-1 support deflection and lightweight summarization.         | 128,000 tokens   |
| **Google**    | `google/gemini-2.0-flash`     | Ultra-fast token delivery with sub-100ms time-to-first-token.                   | 1,000,000 tokens |
| **DeepSeek**  | `deepseek/deepseek-r1`        | Deep step-by-step mathematical, algorithm, and complex logic verification.      | 64,000 tokens    |
| **Groq**      | `groq/llama-3.3-70b`          | Extreme token velocity (>300 tokens/sec) for real-time customer chatbots.       | 128,000 tokens   |

***

## Private On-Premises & Sovereign Model Runtimes

For air-gapped enterprise environments and regulated healthcare/financial deployments, MITHUNAI routes inference queries to internal inference servers without sending prompt tokens over the public internet:

### Connecting a vLLM Cluster

Configure your vLLM endpoint in your deployment environment variables:

```bash .env Configuration theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Points to internal Kubernetes service or private IP
MITHUNAI_LOCAL_INFERENCE_URL="http://vllm-service.inference.svc.cluster.local:8000/v1"
MITHUNAI_LOCAL_INFERENCE_MODEL="meta-llama/Llama-3.3-70B-Instruct"
MITHUNAI_LOCAL_INFERENCE_API_KEY="sk-internal-token"
```

The gateway automatically polls the internal OpenAI-compatible `/v1/models` endpoint, registers the model into the organization's model catalog, and surfaces it inside the [Assistant Studio](/assistants/configure).

***

## Model Selection & Assistant Binding

To bind a specific model to an assistant, provide the model identifier in the configuration payload:

```bash Bind Model via API theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl -X PATCH "https://app.mithunai.com/arukz/api/v1/assistants/$ASSISTANT_ID" \
  -H "Authorization: Bearer $MITHUNAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-3-5-sonnet"
  }'
```

The assistant immediately transitions to the chosen model for all subsequent conversation turns without breaking thread history.