A sitemap helps search engines find the URLs you consider important. It should be boring, clean and machine-readable. One malformed character can stop tools from parsing it correctly.
Common sitemap problems
- Unescaped ampersands or invalid XML entities.
- HTTP, www or query-string versions mixed with canonical URLs.
- Admin, report, data, test or private URLs included by mistake.
- URLs that redirect, 404, noindex or are blocked by robots.txt.
- Duplicate pages with different trailing slash or parameter versions.
Best practice sitemap cleanup
Check this issue on your own website
Reading is useful, but your website data is what matters. Open the matching tool and test it while the problem is fresh.
Use only canonical HTTPS URLs, include pages you want indexed, escape every URL correctly and declare the sitemap in robots.txt. After generating it, test it in a browser and submit it in Google Search Console.
When to regenerate your sitemap
Regenerate the sitemap after adding service pages, publishing blog posts, changing URL structure, removing test folders or fixing canonical redirects. For RankScan AI, the sitemap should include useful tool, service and blog pages while excluding reports and private project files.
Start with the sitemap checker guide or use the sitemap generator.