Case: Sitemap line in robots.txt cut discovery lag

Case: Sitemap line in robots.txt cut discovery lag
A new subdomain's sitemap was submitted only via GSC. New sections sat undiscovered for ~6 days.
What we did:
— Added Sitemap: https://... line to robots.txt for every host
— Pointed each host's robots.txt at its own index file
— Kept GSC submission as a secondary signal
Measured outcome over 4 weeks:
— First-crawl lag for new sections: ~6 days → ~38 hours
— Sitemap discovered across all hosts including non-verified ones
Checklist:
✅ robots.txt names the absolute sitemap URL
✅ One Sitemap line per index file
✅ URL is absolute, not relative
Definition of done: every host's robots.txt declares its sitemap index by absolute URL.