> ## Documentation Index
> Fetch the complete documentation index at: https://docs.37audits.com/llms.txt
> Use this file to discover all available pages before exploring further.
# LLMs.txt Auditor
> Validates the /llms.txt file that guides AI agents through a site's content
**AI Readiness Auditor** — validates the `/llms.txt` file that guides AI agents through a site's content.
Agents get a concise, structured map of the site instead of scraping HTML chrome.
The file follows llms.txt v2 so parsers and agents can read it with fixed markdown rules.
[`/llms.txt`](https://llmstxt.org/) is a markdown file at the site origin that gives AI agents a short project summary and curated links to LLM-friendly pages. This auditor fetches origin `/llms.txt` (required) and `/llms-full.txt` (optional), then checks hosting headers and the v2 format: an H1 name, an optional blockquote, H2 file-list sections, and `[name](url)` links.
## How it works
The auditor requests the origin files and then validates every HTTP 200 response:
1. **Required file** — GET `{origin}/llms.txt`. If the file is missing or not available, the audit is a `FAIL`. An unreachable host is an `ERROR`.
2. **Optional full file** — GET `{origin}/llms-full.txt`. A missing or unreachable full file is ignored; when it returns HTTP 200, the same content-type and format rules apply.
3. **Hosting** — each present file must send `Content-Type: text/plain`, `text/markdown`, or `text/x-markdown` (parameters such as `; charset=UTF-8` are ignored).
4. **Format** — each present file is checked against llms.txt v2: a non-empty H1, a blockquote summary, H2 file-list sections, and markdown links under those sections.
5. **Specification pin** — every run emits `37A-LlmsTxtAuditor-100` (`INFORMATION`) naming llms.txt [v2](https://llmstxt.org/) (10 Aug 2026) and [nbs/index.qmd](https://github.com/AnswerDotAI/llms-txt/blob/6e55a65a7a4b541779dd0585f508717a6ea89a9e/nbs/index.qmd) commit `6e55a65a7a4b541779dd0585f508717a6ea89a9e`. That check does not change overall status.
Any failing rule makes the audit `FAIL`; otherwise a missing blockquote, H2, or link list produces a `WARNING`; otherwise the audit is a `SUCCESS`. An optional BOM at the start of the file is stripped before parsing.
## What it audits
Every row maps to one `checkRule*` method and one `CheckCode`. Content-type and format rules run once per HTTP 200 file (`/llms.txt` and, when present, `/llms-full.txt`). `37A-LlmsTxtAuditor-100` is emitted on every run, including missing-file and fetch errors. CheckCode suffixes use status ranges: `100` information, `200` success, `300–399` warning, `400–499` fail, `500–599` error.
| CheckCode | Status | When it fires | Recommendation |
| --------------------------------------------------------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- |
| `37A-LlmsTxtAuditor-100` | `INFORMATION` | Every run. Records that this auditor implements llms.txt v2 and the pinned `nbs/index.qmd` commit. | — |
| `37A-LlmsTxtAuditor-400` | `FAIL` | Required `/llms.txt` is not available (usually the file is missing). | Create an llms.txt file at `{fileUrl}` so AI agents can discover your site. |
| `37A-LlmsTxtAuditor-401` | `FAIL` | A present file returned a missing or blank `Content-Type` header. | Send Content-Type: text/plain or text/markdown for `{fileUrl}`. |
| `37A-LlmsTxtAuditor-402` | `FAIL` | A present file returned a `Content-Type` other than `text/plain`, `text/markdown`, or `text/x-markdown`. | Serve `{fileUrl}` as text/plain or text/markdown instead of `{observed}`. |
| `37A-LlmsTxtAuditor-403` | `FAIL` | A present file has no H1 title (an empty `#` line is a different rule). | Add an H1 with the project or site name as the first section of `{fileUrl}`. |
| `37A-LlmsTxtAuditor-404` | `FAIL` | A present file has an H1 line whose name is blank (`#` or `# `). | Replace the empty H1 on `{fileUrl}` with the project or site name. |
| `37A-LlmsTxtAuditor-300` | `WARNING` | A present file has no blockquote summary (`>` line) after the H1. | Add a short `> summary` under the H1 on `{fileUrl}`. |
| `37A-LlmsTxtAuditor-301` | `WARNING` | A present file has no H2 file-list sections. | Add H2 sections with markdown file lists on `{fileUrl}`. |
| `37A-LlmsTxtAuditor-302` | `WARNING` | A present file has at least one H2 but no `[name](url)` links under those sections. | Add `- [name](url): notes` entries under the H2 sections of `{fileUrl}`. |
| `37A-LlmsTxtAuditor-500` | `ERROR` | Required `/llms.txt` could not be fetched (connection or I/O failure). | Make `{fileUrl}` reachable over HTTP so the auditor can read llms.txt. |
| `37A-LlmsTxtAuditor-200` | `SUCCESS` | Every rule above passed; one `SUCCESS` check is added with the message `llms.txt at "{llmsTxtUrl}" is valid (H1 "{title}", {sectionCount} H2 section(s), {linkCount} link(s)).` | — |
## Output documentation
| Status | Description | Test logic |
| --------------- | ---------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **INFORMATION** | Provenance only; does not change overall status | Every run includes `37A-LlmsTxtAuditor-100` with llms.txt v2 in the message. `Check.resource` is the pinned `nbs/index.qmd` URL; `recommendation` is `null`. |
| **SUCCESS** | Origin `/llms.txt` exists, is labeled correctly, and matches the v2 format | `/llms.txt` returns HTTP 200 with `text/plain`, `text/markdown`, or `text/x-markdown`. The body has a non-empty H1, a blockquote summary, at least one H2, and markdown links under those sections. Optional `/llms-full.txt` is either absent or also valid. The specification pin check is still present. |
| **WARNING** | The file is present and labeled correctly but is missing useful v2 structure | No blockquote (`300`); no H2 file-list sections (`301`); H2 sections with no markdown links (`302`). The specification pin check is still present. |
| **FAIL** | The required file is missing or violates hosting or H1 rules | `/llms.txt` is not available (`400`); missing Content-Type (`401`); wrong Content-Type (`402`); missing H1 (`403`); empty H1 (`404`). The specification pin check is still present. |
| **ERROR** | The auditor could not reach the required file | Connection or I/O failure fetching `/llms.txt` (`500`). The specification pin check is still present. |
## Risks and considerations
### AI discoverability
* Without `/llms.txt`, agents must scrape HTML (navigation, ads, JavaScript) and guess which pages matter, wasting context and missing the curated path through the site.
* An empty or missing H1 leaves the file without the only required v2 section, so parsers that look for `# {name}` treat the document as incomplete.
### Agent interoperability
* Serving `/llms.txt` as `text/html` (a common 404 or CMS page) causes clients that trust `Content-Type` to skip or mis-parse the file.
* H2 sections without `[name](url)` links give agents headings but no files to fetch, so the overview cannot be followed on demand.
### Optional full file
* `/llms-full.txt` is a community convention, not part of v2. This auditor validates it only when it returns HTTP 200; a missing full file is not a finding.
* v2 no longer gives `## Optional` mechanical meaning. Secondary links may still live under that heading by convention; their absence is not a finding.