Supported Models, LLM Providers & Inference Engines

MITHUNAI integrates with both frontier commercial model providers and sovereign private inference runtimes through an abstracted model catalog. Each assistant connects to a specific model identifier, routing requests through the intelligent inference gateway for semantic caching, rate limiting, and prompt injection defense.

Supported Frontier Models

Frontier models deliver high reasoning precision for complex technical inquiries, code synthesis, and multi-document synthesis:

Provider	Model Identifier	Recommended Use Case	Context Window
Anthropic	`anthropic/claude-3-5-sonnet`	Complex technical reasoning, architecture Q&A, and nuanced code generation.	200,000 tokens
Anthropic	`anthropic/claude-3-5-haiku`	High-volume, low-latency customer support and quick factual lookups.	200,000 tokens
OpenAI	`openai/gpt-4o`	Multimodal comprehension, structured JSON output, and general engineering docs.	128,000 tokens
OpenAI	`openai/gpt-4o-mini`	Cost-effective tier-1 support deflection and lightweight summarization.	128,000 tokens
Google	`google/gemini-2.0-flash`	Ultra-fast token delivery with sub-100ms time-to-first-token.	1,000,000 tokens
DeepSeek	`deepseek/deepseek-r1`	Deep step-by-step mathematical, algorithm, and complex logic verification.	64,000 tokens
Groq	`groq/llama-3.3-70b`	Extreme token velocity (>300 tokens/sec) for real-time customer chatbots.	128,000 tokens

Private On-Premises & Sovereign Model Runtimes

For air-gapped enterprise environments and regulated healthcare/financial deployments, MITHUNAI routes inference queries to internal inference servers without sending prompt tokens over the public internet:

Connecting a vLLM Cluster

Configure your vLLM endpoint in your deployment environment variables:

.env Configuration

# Points to internal Kubernetes service or private IP
MITHUNAI_LOCAL_INFERENCE_URL="http://vllm-service.inference.svc.cluster.local:8000/v1"
MITHUNAI_LOCAL_INFERENCE_MODEL="meta-llama/Llama-3.3-70B-Instruct"
MITHUNAI_LOCAL_INFERENCE_API_KEY="sk-internal-token"

The gateway automatically polls the internal OpenAI-compatible /v1/models endpoint, registers the model into the organization’s model catalog, and surfaces it inside the Assistant Studio.

Model Selection & Assistant Binding

To bind a specific model to an assistant, provide the model identifier in the configuration payload:

Bind Model via API

curl -X PATCH "https://app.mithunai.com/arukz/api/v1/assistants/$ASSISTANT_ID" \
  -H "Authorization: Bearer $MITHUNAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-3-5-sonnet"
  }'

The assistant immediately transitions to the chosen model for all subsequent conversation turns without breaking thread history.

Get started

Knowledge

Assistants & Agents

Channels & Integration

Architecture & Solutions

Administration

Trust and support

Supported Models, LLM Providers & Inference Engines

Supported Frontier Models

Private On-Premises & Sovereign Model Runtimes

Connecting a vLLM Cluster

Model Selection & Assistant Binding

​Supported Frontier Models

​Private On-Premises & Sovereign Model Runtimes

​Connecting a vLLM Cluster

​Model Selection & Assistant Binding

Supported Frontier Models

Private On-Premises & Sovereign Model Runtimes

Connecting a vLLM Cluster

Model Selection & Assistant Binding