How to audit an XML sitemap

1. Define the intended URL set

Start with the pages the client wants discoverable, not every URL a crawler can find. Separate public service and editorial pages from private results, draft previews, filters and obsolete URLs. Write the inclusion rule down so the sitemap can stay correct after the next release.

Google recommends fully qualified URLs in sitemaps. Submitting a sitemap is a discovery hint, not proof of crawling or indexing. Use it alongside working internal links and consistent canonical signals.

• Confirm the preferred HTTPS hostname.
• Find sitemap declarations in robots.txt and the submitted sitemap in Search Console, if accessible.
• List any sitemap indexes and their child files.
• Record which site sections should be included and excluded.

2. Fetch and inspect the file

Open the sitemap anonymously. Record the HTTP status, final URL and whether the response is XML rather than a login page or application error. Parse it with an XML-aware tool instead of copying lines from the browser. Keep the original file with a timestamp for the audit record.

• Check child sitemap files as well as the index.
• Check that locations are absolute URLs and match the intended host.
• Identify duplicate entries and unexpected query-string variants.
• Inspect last-modified dates only if supplied. Do not assume a fresh timestamp means content changed.

Google uses lastmod when it is consistently accurate. Generate it from a meaningful page update, or omit it when that date is unknown; do not stamp every page with the time the sitemap was built.

3. Compare listed URLs with live responses

For a small site, inspect every entry. For a larger site, sample each template and section, document the sample, then expand it when a pattern appears. Request the exact listed URL before following redirects. Inspect the final page's canonical and both HTML and HTTP indexing directives.

4. Fix the generator, not just the XML

Trace each bad entry to the CMS field, route catalog or generation rule that produced it. Editing a generated file by hand can be undone at the next build. Preview the removed and added URLs, approve the intended set, and then publish the change through the site's normal release process.

• Save the before-and-after URL lists.
• Verify intended public pages remain included.
• Re-fetch the hosted sitemap and representative pages after the release.
• Confirm robots.txt references the current sitemap location.
• Keep the previous release or generator configuration available for rollback.

5. Record discovery separately from indexing

Use Search Console's Sitemaps report to monitor retrieval and processing. A successful fetch does not establish that every listed URL is indexed. Track page-level observations separately and allow time for new data. On a new site, 'no data yet' is a valid baseline.