Skip to main content
SEO Auditor — checks whether a nonexistent page returns a proper 404/410 status and a helpful custom 404 page.

SEO

A noindexed, properly-statused 404 page keeps search engines from indexing junk URLs.

User experience

Helpful content, navigation, and a search box turn a dead end into a recoverable page.
This auditor requests a synthetic, never-existing path on the audited host, then evaluates the response independently: the HTTP status code must be 404 or 410 (not a soft 404), and — only once the status code is correct — the page itself is checked for custom not-found content, a noindex meta tag, navigation links, other helpful links, and a search box.

​
How it works

  1. Generate a test URL — appends /this-page-should-not-exist-<timestamp> to the audited host so the path is guaranteed not to exist.
  2. Fetch — requests the test URL with jsoup (ignoreHttpErrors, redirects disabled, HTTP Basic Auth when provided) in a single request.
  3. Status rules — a 200 response (soft 404) or any status other than 200/404/410 fails immediately; the page content is not evaluated in either case.
  4. Content rules — once the status code is confirmed 404 or 410, the page body is checked for custom not-found messaging (FAIL if missing), then noindex meta tag, navigation links, helpful links, and a search box (each WARNING if missing).
  5. Aggregate — any FAIL rule wins over WARNING; if every rule passes, the audit is SUCCESS. Fetch errors (invalid URL, unreachable host) produce an ERROR response.

​
What it audits

Every row maps to one checkRule* method and one CheckCode. CheckCode suffixes use status ranges: 200 success, 300–399 warning, 400–499 fail, 500–599 error. Content rules (300–303, 402) are only evaluated once the status code rules (400, 401) have passed.

​
Output documentation

​
Risks and considerations

  • A nonexistent page that returns HTTP 200 looks fine to a casual visitor but is invisible to tools that rely on status codes — broken-link checkers, uptime monitors, and search engine crawlers all misread it as a valid page.
  • Soft 404s can also get indexed and compete with real pages in search results, diluting relevance signals for the site.

​
Indexed error pages waste crawl budget and confuse rankings

  • Without a noindex tag, search engines may crawl and index thousands of unique 404 URLs (especially on sites with many broken inbound links), wasting crawl budget that could go to real content.

​
A dead end costs a visitor, not just a page view

  • A blank or generic 404 page gives a lost visitor nothing to do but leave. Custom messaging, navigation, helpful links, and search all give them a path back into the site instead of bouncing.

​
What this auditor does not check

  • It does not verify that specific known-bad or previously-existing URLs return 404; it only tests a freshly generated, guaranteed-nonexistent path.
  • It does not check HTTP→HTTPS redirects, canonical tags, or sitemap hygiene for stale URLs — those are covered by other auditors.