> ## Documentation Index > Fetch the complete documentation index at: https://docs.mithunai.com/llms.txt > Use this file to discover all available pages before exploring further. # Connect a website, sitemap or GitHub > Point MITHUNAI at a documentation website, an XML sitemap or a GitHub repository, and control which pages and paths it reads with include and exclude patterns. A source tells MITHUNAI where to read content from. Create it once, then sync it whenever the content changes. For files on your computer, see [File uploads](/knowledge/file-uploads). 1. Open **Knowledge** and select **Connect a knowledge source**. 2. Choose **Website**, **Sitemap**, **GitHub** or **File upload**, and fill in the fields described below. 3. Save. The source appears in the list and its first ingestion starts; its status updates as the job runs. Use **Re-sync** to ingest changes, and **Remove** to delete a source. ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} curl --request POST "$MITHUNAI_URL/arukz/api/v1/knowledge/sources" \ --header "Authorization: Bearer $MITHUNAI_API_KEY" \ --header "Content-Type: application/json" \ --data '{ "collection_id": "", "source_type": "website", "name": "Product documentation", "configuration": { "seed_url": "https://docs.example.com/", "path_prefix": "/docs", "max_depth": "3", "max_pages": "500" } }' ``` Then [sync the source](/api-reference/knowledge/sync-source) to ingest it. Every `configuration` value is a string, and path lists are comma-separated inside one string. See [Create a source](/api-reference/knowledge/create-source). ## Sitemap Reads exactly the URLs listed in a `sitemap.xml`, without following links. This is the most predictable way to ingest a documentation site: you ingest what you publish, nothing more. | Setting | Required | Description | | ----------------------------------------------- | -------- | --------------------------------------------------------------------------------------------- | | `seed_url` | Yes | The sitemap URL, such as `https://docs.example.com/sitemap.xml`. Sitemap indexes are followed | | `max_pages` | No | Maximum pages read per run, from 1 to 5,000. Default 500 | | `path_prefix`, `include_paths`, `exclude_paths` | No | Filter the listed URLs, as for a website | Listed URLs must be on the same origin as the sitemap. ## Website Crawls a site from a start URL, following links on the same origin (scheme, host and port). Use it when no sitemap exists. | Setting | Required | Description | | --------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------- | | `seed_url` | Yes | Where the crawl starts. An absolute `http` or `https` URL | | `path_prefix` | No | Only crawl paths at or below this prefix, such as `/docs`. Matches whole path segments, so `/docs` does not match `/docs-internal` | | `max_depth` | No | How many links to follow from the start URL, from 1 to 10. Default 3 | | `max_pages` | No | Maximum pages per run, from 1 to 5,000. Default 500 | | `include_paths` | No | Comma-separated path patterns. When set, only matching pages are ingested | | `exclude_paths` | No | Comma-separated path patterns to skip. Exclusions win over inclusions | Redirects are not followed, and links to images, media, archives, fonts, stylesheets and scripts are skipped. ## GitHub Reads files from a repository at a branch, tag or commit. | Setting | Required | Description | | --------------- | -------- | ---------------------------------------------------------------------------------------------------- | | `owner` | Yes | The GitHub account or organisation, such as `acme` | | `repository` | Yes | The repository name, such as `platform-docs` | | `ref` | No | Branch, tag or commit SHA. Empty follows the default branch | | `include_paths` | No | Comma-separated path patterns, such as `docs/,README.md`. When set, only matching files are ingested | | `exclude_paths` | No | Comma-separated patterns to skip. **Replaces** the default list rather than adding to it | By default MITHUNAI skips `.git/`, `node_modules/`, `vendor/`, `dist/`, `build/`, `target/`, `.venv/`, `__pycache__` directories, minified JavaScript and lock files. If you set `exclude_paths`, include any of those you still want skipped. Repositories are read with the GitHub access your MITHUNAI deployment is configured with. To connect a private repository, ask your MITHUNAI administrator to grant that access. You never paste a token into a source. ## Path patterns In `include_paths` and `exclude_paths`: | Pattern | Matches | | -------------- | -------------------------------- | | `*` | Anything within one path segment | | `**` | Anything across segments | | `?` | One character | | A trailing `/` | Everything below a directory | For example, `docs/**/*.md,README.md` ingests Markdown under `docs/` and the root README. ## Choose the right source | If your content is | Use | | -------------------------------------- | --------------------------------------------------- | | A documentation site with a sitemap | **Sitemap** | | A site without a sitemap | **Website**, with `path_prefix` to stay in the docs | | Docs-as-code in a repository | **GitHub**, with `include_paths` | | PDFs, Word documents or exported files | [File upload](/knowledge/file-uploads) | ## Security Source settings are visible to every member of your organisation, so they never hold credentials. Crawls cannot reach private or internal network addresses, and URLs containing a username or password are refused. See [Security](/security).