Run an answer-quality eval on an assistant
Ask an assistant up to 10 known questions and score each answer for grounding, citations, expected sources and facts, and correct abstention.
POST
/
assistants
/
{assistant_id}
/
evals
Run an answer-quality eval on an assistant
curl --request POST \
--url https://app.mithunai.com/mithunai/api/v1/assistants/{assistant_id}/evals \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data '
{
"cases": [
{
"question": "<string>",
"expected_facts": [
"<string>"
],
"expected_sources": [
"<string>"
],
"should_abstain": true
}
]
}
'import requests
url = "https://app.mithunai.com/mithunai/api/v1/assistants/{assistant_id}/evals"
payload = { "cases": [
{
"question": "<string>",
"expected_facts": ["<string>"],
"expected_sources": ["<string>"],
"should_abstain": True
}
] }
headers = {
"Authorization": "Bearer <token>",
"Content-Type": "application/json"
}
response = requests.post(url, json=payload, headers=headers)
print(response.text)const options = {
method: 'POST',
headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
body: JSON.stringify({
cases: [
{
question: '<string>',
expected_facts: ['<string>'],
expected_sources: ['<string>'],
should_abstain: true
}
]
})
};
fetch('https://app.mithunai.com/mithunai/api/v1/assistants/{assistant_id}/evals', options)
.then(res => res.json())
.then(res => console.log(res))
.catch(err => console.error(err));{
"status": "<string>",
"problem": {},
"message": "<string>",
"total": 123,
"passed": 123,
"pass_rate": {},
"results": [
{}
]
}Asks the assistant each case through its normal answer path and scores what comes back by comparing text, never with a model. See Answer quality evals for what each check means. Each case counts as one answer against your monthly allowance; no conversation is created and nothing is stored. Requires the owner, admin or editor role.
With no model provider configured,
status is cannot_run and nothing is asked. When the model or your allowance runs out partway, the cases scored before it are returned.
string
required
The assistant to test (UUID). Another organization’s assistant returns
404.object[]
required
Response
string
required
done, or cannot_run. Treat a value you do not recognise as cannot_run.string | null
required
no_model, model_unavailable or allowance_reached when it could not run.string
required
Why it could not run, in a sentence. Empty when it ran.
integer
required
Cases scored.
integer
required
Cases that passed.
number | null
required
passed / total, or null with no cases scored.object[]
required
One per case, in order:
question, should_abstain, abstained, answer, sources (title,
url, snippet), groundedness (0 to 1, or null when it declined), grounded,
citations_correct, source_hit (null when the case names no source), facts_found,
facts_missing, abstention_correct and passed.Last modified on November 12, 2026
Run an answer-quality eval on an assistant
curl --request POST \
--url https://app.mithunai.com/mithunai/api/v1/assistants/{assistant_id}/evals \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data '
{
"cases": [
{
"question": "<string>",
"expected_facts": [
"<string>"
],
"expected_sources": [
"<string>"
],
"should_abstain": true
}
]
}
'import requests
url = "https://app.mithunai.com/mithunai/api/v1/assistants/{assistant_id}/evals"
payload = { "cases": [
{
"question": "<string>",
"expected_facts": ["<string>"],
"expected_sources": ["<string>"],
"should_abstain": True
}
] }
headers = {
"Authorization": "Bearer <token>",
"Content-Type": "application/json"
}
response = requests.post(url, json=payload, headers=headers)
print(response.text)const options = {
method: 'POST',
headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
body: JSON.stringify({
cases: [
{
question: '<string>',
expected_facts: ['<string>'],
expected_sources: ['<string>'],
should_abstain: true
}
]
})
};
fetch('https://app.mithunai.com/mithunai/api/v1/assistants/{assistant_id}/evals', options)
.then(res => res.json())
.then(res => console.log(res))
.catch(err => console.error(err));{
"status": "<string>",
"problem": {},
"message": "<string>",
"total": 123,
"passed": 123,
"pass_rate": {},
"results": [
{}
]
}