> ## Documentation Index
> Fetch the complete documentation index at: https://docs.37audits.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Sitemap
> Finds the XML sitemap through robots.txt or the usual paths and checks that it parses and was updated recently.
**SEO Auditor** — locates the XML sitemap of the audited site (through `robots.txt` or the usual paths), parses it and checks that its latest `` is recent.
Reads `Sitemap:` lines in `robots.txt`, then tries `/sitemap.xml`, `/sitemap_index.xml` and `/sitemap/sitemap.xml`.
Follows the Sitemaps XML format, protocol version 0.9, published at sitemaps.org.
An XML sitemap tells search engines which pages a site has and when each one last changed. Crawlers find it through the `Sitemap:` line of `robots.txt` or at a well-known path, and use its `` dates to decide what to recrawl. A missing sitemap leaves discovery to link following; a sitemap whose dates stopped moving tells crawlers nothing new is there. This auditor never fetches the audited page itself: it only reads `robots.txt` and the candidate sitemaps.
## How it works
1. **Site root** — takes `scheme://host[:port]` from the audited URL. A URL without a scheme or hostname ends the audit with `501`.
2. **robots.txt** — fetches `/robots.txt` (as 37AuditsBot, with the site's Basic Auth when configured) and collects every `Sitemap:` line. The probe never calls `ensureNotBlocked`: a 403 or any other answer on `robots.txt` is recorded in `issues`, not treated as a blocked audit.
3. **Candidates** — the `robots.txt` URLs, or when there are none, `/sitemap.xml`, `/sitemap_index.xml` and `/sitemap/sitemap.xml` under the site root.
4. **Parse** — fetches each candidate in turn (redirects followed, bodies up to 50 MB, 30 s per request, gzip detected by magic bytes even without `Content-Encoding`) until one answers HTTP 2xx with a `` or `` root. Child sitemaps of an index are not fetched. The entries' `` values are parsed (W3C Datetime with or without time zone, date-only read as end of that day) and the latest one is kept; values that do not parse are listed in `issues`.
5. **Rules** — no candidate parsed fails (`400`). On the parsed sitemap, a latest `` older than the freshness window warns (`300`), and no parsable `` at all warns (`301`). Independently, a `robots.txt` that answered HTTP 200 without a `Sitemap:` line warns (`302`), even when a fallback path yielded a sitemap.
6. **Freshness window** — `SITEMAP_MAX_AGE_DAYS` (Lambda environment, default 7 days).
7. **Aggregate** — FAIL wins over WARNING; a sitemap whose latest `` is within the window emits `200`, and the audit is `SUCCESS` when no warning fired.
8. **Specification pin** — every run emits `37A-SitemapAuditor-100` (`INFORMATION`), naming the [Sitemaps XML format, protocol version 0.9](https://www.sitemaps.org/protocol.html). It does not change the overall status.
## What it audits
Every row maps to one `checkRule*` method and one `CheckCode`, in the order the auditor runs them. CheckCode suffixes use status ranges: `100` information, `200` success, `300–399` warning, `400–499` fail, `500–599` error.
| CheckCode | Status | When it fires | Recommendation | Why it matters |
| - | - | - | - | - |
| `37A-SitemapAuditor-100` | `INFORMATION` | Always. Records the Sitemaps protocol version this auditor implements. | — (recommendation is `null`; nothing for the site to change) | — |
| `37A-SitemapAuditor-400` | `FAIL` | No candidate sitemap (from `robots.txt` or the fallback paths) could be fetched and parsed as a `` or ``. | Publish an XML sitemap at `{siteBase}`/sitemap.xml (or any URL) that answers HTTP 200 with a `` or `` root, and reference it from `{robotsTxtUrl}` with a "Sitemap:" line. Candidates tried: `{candidates}`. | Without a sitemap, search engines only discover pages by following links, so deep or new pages are indexed late or not at all and bring in no search traffic. |
| `37A-SitemapAuditor-300` | `WARNING` | The latest `` of the parsed sitemap is older than the freshness window (`SITEMAP_MAX_AGE_DAYS`, default 7 days). Resource is the sitemap URL. | Regenerate `{sitemapUrl}` whenever pages change so its `` values stay within `{maxAgeDays}` days; the latest is currently `{latestLastmod}`. | Search engines use lastmod to decide which pages to recrawl; a sitemap that looks abandoned is trusted less, so new and updated pages are found and ranked later. |
| `37A-SitemapAuditor-301` | `WARNING` | The sitemap parsed but none of its entries has a parsable ``, so freshness cannot be assessed. Resource is the sitemap URL. | Add a `` element in W3C Datetime format (for example 2026-10-16 or 2026-10-16T09:30:00Z) to each `` (or ``) entry of `{sitemapUrl}`, set to the date the page last changed. | Without lastmod dates, search engines cannot tell which pages changed and crawl them all alike, so updates to important pages are picked up later. |
| `37A-SitemapAuditor-302` | `WARNING` | `robots.txt` answered HTTP 200 but lists no `Sitemap:` line. Fires in addition to whatever the fallback paths yielded. Resource is the `robots.txt` URL. | Add the line "Sitemap: `{sitemapUrl}`" to `{robotsTxtUrl}`. | robots.txt is the first file every crawler reads, and a Sitemap: line there is how search engines discover the sitemap without a manual submission; without it some crawlers never find the sitemap at all. |
| `37A-SitemapAuditor-200` | `SUCCESS` | A sitemap parsed and its latest `` is within the freshness window. Message names the sitemap URL, its type, entry count, latest `` and the window. Resource is the sitemap URL. | — | — |
| `37A-SitemapAuditor-501` | `ERROR` | The audited URL is missing, malformed or has no scheme or hostname, so no site root can be built. | Audit a URL with a scheme and a hostname, such as [https://example.com/](https://example.com/), instead of `{url}`. | — |
## Output documentation
| Status | Description | Test logic |
| - | - | - |
| **SUCCESS** | A sitemap was found and is fresh | A candidate parsed and its latest `` is within `SITEMAP_MAX_AGE_DAYS` (`200`), and `robots.txt` either lists it or is not served. |
| **WARNING** | A sitemap was found but needs attention | Latest `` older than the window (`300`); no parsable `` (`301`); `robots.txt` served without a `Sitemap:` line (`302`). |
| **FAIL** | No sitemap | No candidate answered HTTP 2xx with a `` or `` root (`400`); the candidates and the per-candidate issues (HTTP status, empty XML, unknown root, fetch error) are in the check data. |
| **ERROR** | The auditor could not run | The audited URL has no scheme or hostname (`501`). |
| **INFORMATION** | Specification pin | Always emitted (`100`); does not change overall status. |
Every check carries the same `data` map: `siteBase`, `robotsTxtUrl`, `robotsTxtHttpStatus`, `robotsTxtListsSitemap`, `candidateSitemaps`, `selectedSitemapUrl`, `sitemapType` (`urlset` or `sitemapindex`), `sitemapHttpStatus`, `contentType`, `entryCount`, `latestLastmod` (ISO instant or `null`), `maxAgeDays` and `issues`.
## Risks and considerations
### Why the sitemap matters
* Search engines discover and prioritise pages from the sitemap; a site without one relies on internal links alone, and pages far from the home page may never be crawled.
* `` is the signal crawlers use to recrawl changed pages first. Dates that never move, or are missing, make every page look equally stale.
* Listing the sitemap in `robots.txt` is the one discovery mechanism every crawler shares; Search Console submissions only reach Google.
### Freshness window
* The window is `SITEMAP_MAX_AGE_DAYS` (default 7 days). A brochure site that rarely changes will warn with `300` even though nothing is wrong with its sitemap; the warning is a prompt to keep `` honest, not a defect of the site.
* Date-only `` values (`2026-10-16`) are read as the end of that day in UTC, so a sitemap updated today is always inside the window.
### What this auditor does not check
* Only the first sitemap that parses is examined. Child sitemaps of a `` are not fetched, so their own `` values and URL counts are not seen.
* The URLs listed in the sitemap are not fetched or validated, and the 50,000-URL and 50 MB limits of the protocol are not enforced.
* Sitemaps in other formats (RSS, Atom, plain-text URL lists) are reported as an unknown root element and do not count.
* A `robots.txt` that is missing (404) or blocked (403) is not a finding by itself; the auditor just falls back to the usual paths.
### Changes from the previous version
* An invalid audited URL (no scheme or hostname) is now an `ERROR` with `37A-SitemapAuditor-501`; it used to be a `FAIL` with no check.
* A sitemap without any parsable `` now gets its own warning, `37A-SitemapAuditor-301`; it used to be reported as "out of date" together with the stale case.
* A `robots.txt` that answers HTTP 200 without a `Sitemap:` line now warns with `37A-SitemapAuditor-302`, so a site whose sitemap was only found at a fallback path is a `WARNING` rather than a `SUCCESS`.
* The `available` field is gone from the check data; `selectedSitemapUrl` is `null` when no sitemap parsed.
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.