XML Sitemap Generator

Why generate a sitemap from a crawl?

A sitemap built from a live crawl reflects what a search engine can actually reach by following your links, which is precisely the point. Pages that exist in the CMS but are linked from nowhere will not appear, and that is a feature, not a bug: if our crawler cannot find a page, Googlebot will struggle too. Fix the internal link first, then regenerate.

What the generator checks on every page

For each discovered URL the tool records the HTTP status, honours noindex meta tags and X-Robots-Tag headers, reads the Last-Modified header where the server provides one (it becomes the <lastmod> value), and normalises the URL by stripping fragments and query strings. The output is standards-compliant XML that validates against the sitemaps.org schema, ready for Search Console.

When you should not use a generated file

If your site runs on a CMS that can serve a dynamic sitemap, use that instead: it updates itself when you publish. A hand-uploaded static file is right for hand-coded HTML sites, legacy stacks with no plugin ecosystem, and quick audits where you want to count reachable pages. It is also a fast way to see how much of a site survives after a migration; compare the URL count before and after.

Sitemap myths worth ignoring

Submitting a sitemap does not guarantee indexing; Google treats it as a hint, and low-quality pages stay out regardless. Priority and changefreq fields are ignored by Google, which is why this generator does not emit them. And a sitemap does not need to list every URL variant, only the canonical ones you actually want in search.