Skip to main content
AI Readiness Auditor — validates that the site serves HTML to browsers and markdown to AI clients via Accept content negotiation.

AI Readiness

AI agents that ask for markdown get clean markdown source instead of a full HTML page.

Compatibility

Browsers keep receiving HTML by default; markdown is only served when explicitly requested.
Large language models and AI agents consume markdown far more efficiently than rendered HTML: it is smaller, keeps the document structure, and drops navigation, scripts, and styling noise. A growing convention is to serve the same URL as markdown when the client sends Accept: text/markdown, while browsers — which send Accept: text/html — keep receiving the normal HTML page. This auditor requests the audited URL both ways and verifies each response’s Content-Type and body.

​
How it works

The auditor issues two GET requests to the audited URL (following redirects, with HTTP Basic Auth when provided):
  1. Browser request — sends Accept: text/html and expects a Content-Type of text/html or application/xhtml+xml.
  2. AI-client request — sends Accept: text/markdown and expects a Content-Type of text/markdown or text/x-markdown.
  3. Body sanity check — when the markdown response declares a markdown media type, the body must not start as a full HTML document (<!DOCTYPE html or <html).
  4. Aggregate — any failing rule makes the audit FAIL; when every rule passes, the audit is SUCCESS.
Media type parameters such as ;charset=UTF-8 are ignored when comparing content types. An unreachable URL produces an ERROR response; a non-200 status on either request produces a FAIL without evaluating the rules.

​
What it audits

Every row maps to one checkRule* method and one CheckCode.

​
Output documentation

​
Risks and considerations

​
AI readiness

  • Agents and LLM crawlers that request text/markdown and receive a full HTML page must parse markup, scripts, and navigation chrome, wasting context window and degrading answer quality.
  • Serving markdown through Accept negotiation keeps a single canonical URL per page — no separate .md URLs to publish, crawl, or keep in sync.

​
Browser compatibility

  • Content negotiation must never leak markdown to browsers: a markdown response to Accept: text/html renders as plain text and breaks the page for every visitor.
  • Responses that vary by Accept should include Vary: Accept so caches and CDNs do not serve the markdown variant to browsers or vice versa.

​
Correct labeling

  • A missing Content-Type forces clients to sniff the payload, which is unreliable and a known security concern.
  • Declaring text/markdown while returning an HTML body is worse than not negotiating at all: clients trust the label and feed HTML into markdown pipelines.