Skip to main content
An assistant answers only from the knowledge attached to it. Content gets there through a source — a website, a sitemap, a GitHub repository or uploaded files — which is synced into a collection. Syncing runs an ingestion job in the background: MITHUNAI fetches the content, extracts its text, splits it into passages and indexes them. A collection is what one or more assistants answer from, and a passage is what a citation points at.

​
The building blocks

A collection is a corpus: the set of content one or more assistants answer from. Every source ingests into a collection.A collection’s embedding model is fixed when the collection is created and never changes, because passages embedded by different models cannot be compared. To move to a different embedding model, create a new collection and ingest your sources into it.Collections can be renamed, archived and restored. An archived collection keeps serving answers but accepts no new ingestion. Collections cannot be deleted.
A source is a standing definition of where content comes from: a website, a sitemap, a GitHub repository or a set of uploaded files. Syncing it again picks up what changed. See Knowledge sources.
Syncing a source starts an ingestion job that runs in the background. You can follow its progress and cancel it. See Ingestion.
A document is one ingested item, such as a page or a file. Documents are split into passages, and passages are what retrieval searches and what citations point at, so a citation lands on the paragraph that supports a claim rather than on a 40-page manual.

​
Supported content

MITHUNAI extracts text from HTML, Markdown, plain text, reStructuredText, source code, PDF and Word documents. Content it cannot extract is skipped rather than ingested badly.

​
Your content is treated as untrusted input

Anyone who can edit your documentation can put text into your corpus. For a public site or repository, that can include anyone whose pull request you merge. MITHUNAI therefore:
  • Bounds every fetch, so a runaway or hostile source cannot consume unbounded resources.
  • Restricts where crawls can go, so a link or redirect cannot reach internal infrastructure.
  • Detects and redacts secrets in extracted content, so a credential committed to a repository is removed before an assistant can repeat it.
  • Separates content from instructions when answering, so text such as “ignore your instructions” is treated as content.
These measures reduce prompt-injection risk; no system that feeds untrusted text to a model can eliminate it. Review what you connect as part of your security perimeter.

​
Next

Connect a source

Websites, sitemaps and GitHub repositories.

Upload files

PDFs, Word documents, Markdown and text.
Last modified on September 26, 2026