Skip to main content
MITHUNAI connects to sixteen kinds of source. In the console they are thirteen choices under four headings; GitHub is one choice that reads four different things from a repository. There are no other connectors: Google Drive, S3, GitLab, Linear, Slack and Discord are not supported, and neither is a Zendesk help centre or a Discourse forum that needs signing in. Five kinds always sign in with a saved connection, your own token for that service, stored encrypted: Confluence, Notion, Jira, Jira Service Management and Zendesk tickets. The four GitHub kinds use one only for a private repository (Connect a private GitHub repository). Everything a connection can read, your assistant can quote, so read Saved connections before you save one. No other source takes a credential: you never paste a token into a source’s settings, and help centres and forums are read as a signed-out visitor reads them. A GitHub source without a connection reads public repositories with the access your MITHUNAI deployment is configured with.

How a source is read

Every source goes through the same pipeline:
  • Fetching is bounded and safe. Crawls stay on the start URL’s origin, do not follow redirects, skip media and binary links, and cannot reach private or internal network addresses.
  • Credentials are redacted before anything is stored or embedded, so a key committed to a repository does not become searchable.
  • Passages follow structure. Each passage keeps the heading trail and section anchor its citation points at.
The crawler does not run JavaScript, so a page that renders its content only in the browser yields little text. Publish a server-rendered or static version, or use a sitemap of pages that are.

Keeping sources up to date

A source is read when it is synced. You sync it; a deployment can also re-sync sources on a schedule, and that is off unless your deployment’s operator has turned it on. MITHUNAI does not receive push events from GitHub or any other system.
  • Trigger Sync a source from the pipeline that publishes your content. Command line has a CI example. This works on every deployment and is the only way to pick a change up straight away.
  • A sync re-reads the whole source. Documents whose content has not changed are counted as documents_unchanged and are not re-indexed, so a re-sync mostly costs the fetch.
  • For a GitHub source, a file whose Git revision has not changed is not fetched again.

Scheduled re-sync

Scheduled re-sync is a setting of the deployment, not of your organisation or of a source. Ask your MITHUNAI contact whether it is on for yours; until you know, treat syncing as yours to trigger. When it is on:
  • One interval for every source. A source is re-read once the deployment’s interval, 24 hours by default, has passed since its last run. There is no per-source schedule, and nothing in the console sets one.
  • Every source type except two. File uploads and custom answers are never re-synced on a schedule: neither has anything upstream to re-read.
  • Only sources that are active and have been synced once. A paused source is skipped, a source in an archived collection is skipped, and a source nobody has synced is not started.
  • Unchanged sources are not re-read. A scheduled run first checks whether the source has moved, such as a repository’s latest commit, and ends skipped when it has not. A website or sitemap cannot be checked without crawling it, so it is always re-read; unchanged pages still cost nothing to re-index.
  • With the access of whoever last synced it. A scheduled run acts as the person or API key that last asked for that source to be synced. If that person has since been removed, or that key revoked or expired, the source is skipped until someone syncs it by hand.
See Ingestion for job statuses, retries and cancellation.

Content in a system MITHUNAI cannot read

Export it to a format MITHUNAI does read, then connect that: Re-export and re-upload when the content changes; an upload source does not track the original.
Last modified on November 7, 2026