> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mithunai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Automated Grounding Benchmarks & Hallucination Auditing

> How MITHUNAI measures factual grounding, evaluates answer faithfulness, detects hallucinations, and executes automated golden test suites before production deployment.

MITHUNAI evaluates answer quality using automated factual grounding benchmarks that verify every generated sentence against retrieved source passages before deployment. Unlike subjective human spot-checks, our evaluation framework runs quantitative scoring across three rigorous metrics: factual faithfulness, citation recall, and abstention precision on unanswerable negative queries.

```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart LR
    TEST["Golden Evaluation Suite\n(Questions + Ground Truth)"]
    RUN["MITHUNAI Evaluation Engine"]
    SCORES["Automated Scorecard\n(Faithfulness, Citation Accuracy, Abstention)"]

    TEST --> RUN --> SCORES
```

***

## Core Grounding Metrics

The platform evaluates assistants against four empirical metrics:

| Metric                      | What It Measures                                                                             | Target Threshold |
| :-------------------------- | :------------------------------------------------------------------------------------------- | :--------------- |
| **Faithfulness Score**      | Percentage of claims in the answer that are logically entailed by retrieved passages.        | $\ge 99.0\%$     |
| **Citation Precision**      | Accuracy of source URLs and line ranges cited for specific factual assertions.               | $\ge 98.5\%$     |
| **Abstention Precision**    | Accuracy of declining to answer queries when required information is absent from the corpus. | $\ge 99.5\%$     |
| **Answer-First Compliance** | Whether the primary conclusion or solution is stated in the lede sentence.                   | $100\%$          |

***

## Running Golden Dataset Evaluations

A **Golden Dataset** consists of paired test items containing realistic user questions, required source passages, and expected canonical answers:

```json Golden Test Item Schema theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "test_id": "eval_pay_01",
  "question": "What is the maximum file size for webhook payload delivery?",
  "expected_behavior": "answer",
  "required_citation_urls": ["https://docs.example.com/webhooks/limits"],
  "expected_keywords": ["10 MB", "compressed", "413 Payload Too Large"]
}
```

### Testing Negative / Out-of-Scope Queries

Crucially, 30% of your evaluation suite should consist of **negative queries**—questions that resemble valid inquiries but whose answers are deliberately missing from your documentation:

```json Negative Test Item Schema theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "test_id": "eval_neg_04",
  "question": "How do I configure GraphQL subscriptions for legacy v0.9 endpoints?",
  "expected_behavior": "abstain",
  "prohibited_behavior": "guess_or_hallucinate",
  "expected_abstention_flag": true
}
```

If the assistant attempts to guess or generate hypothetical GraphQL schemas instead of abstaining, the evaluation gate fails automatically.

***

## CI/CD Regression Testing

Automate grounding verification in your build pipeline using the MITHUNAI CLI:

```bash Automated Benchmark Run theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Run evaluation suite against test assistant
mithunai eval run \
  --assistant asst_staging_01 \
  --dataset ./tests/golden-questions.json \
  --min-faithfulness 0.98 \
  --output ./eval-report.json

# Exit code 0 if all tests pass; 1 if regressions are detected
echo "Evaluation passed successfully."
```

By enforcing these evaluation gates in CI/CD, documentation and engineering teams ensure that new documentation syncs never introduce factual regressions into customer-facing assistants.