Skip to main content
AI Readiness Auditor — validates the /llms.txt file that guides AI agents through a site’s content.

AI Readiness

Agents get a concise, structured map of the site instead of scraping HTML chrome.

Interoperability

The file follows llms.txt v2 so parsers and agents can read it with fixed markdown rules.
/llms.txt is a markdown file at the site origin that gives AI agents a short project summary and curated links to LLM-friendly pages. This auditor fetches origin /llms.txt (required) and checks hosting headers and the v2 format: an H1 name, an optional blockquote, H2 file-list sections, and [name](url) links. It also checks the hosting headers and H1 of /llms-full.txt (optional) unless the Evaluate llms-full.txt preference is turned off.

How it works

The auditor requests the origin files and then validates every HTTP 200 response:
  1. Required file — GET {origin}/llms.txt. If the file is missing or not available, the audit is a FAIL. An unreachable host is an ERROR.
  2. Optional full file — when Evaluate llms-full.txt is on (the default), GET {origin}/llms-full.txt. A missing or unreachable full file is ignored; when it returns HTTP 200, the content-type and H1 rules apply. The blockquote, H2 and link rules do not, because the full file carries page content rather than curated file lists. When the preference is off, the file is not requested.
  3. Hosting — each present file must send Content-Type: text/plain, text/markdown, or text/x-markdown (parameters such as ; charset=UTF-8 are ignored).
  4. Format — /llms.txt is checked against llms.txt v2: a non-empty H1, a blockquote summary, H2 file-list sections, and markdown links under those sections. /llms-full.txt is checked for the H1 only.
  5. Specification pin — every run emits 37A-LlmsTxtAuditor-100 (INFORMATION) naming llms.txt v2 (10 Aug 2026) and nbs/index.qmd commit 6e55a65a7a4b541779dd0585f508717a6ea89a9e. That check does not change overall status.
Any failing rule makes the audit FAIL; otherwise a missing blockquote, H2, or link list produces a WARNING; otherwise the audit is a SUCCESS. An optional BOM at the start of the file is stripped before parsing.

What it audits

Every row maps to one checkRule* method and one CheckCode. Content-type and H1 rules (401–404) run once per HTTP 200 file (/llms.txt and, when evaluated and present, /llms-full.txt); the structure warnings (300–302) run on /llms.txt only. 37A-LlmsTxtAuditor-100 is emitted on every run, including missing-file and fetch errors. CheckCode suffixes use status ranges: 100 information, 200 success, 300–399 warning, 400–499 fail, 500–599 error.

Output documentation

Preferences

Risks and considerations

AI discoverability

  • Without /llms.txt, agents must scrape HTML (navigation, ads, JavaScript) and guess which pages matter, wasting context and missing the curated path through the site.
  • An empty or missing H1 leaves the file without the only required v2 section, so parsers that look for # {name} treat the document as incomplete.

Agent interoperability

  • Serving /llms.txt as text/html (a common 404 or CMS page) causes clients that trust Content-Type to skip or mis-parse the file.
  • H2 sections without [name](url) links give agents headings but no files to fetch, so the overview cannot be followed on demand.

Optional full file

  • /llms-full.txt is a community convention, not part of v2. This auditor validates its Content-Type and H1 only when it returns HTTP 200, and skips it entirely when Evaluate llms-full.txt is off; a missing full file is not a finding.
  • v2 no longer gives ## Optional mechanical meaning. Secondary links may still live under that heading by convention; their absence is not a finding.