Hreflang's hidden crawl-budget cost on large multilingual estates

Hreflang's hidden crawl-budget cost on large multilingual estates

An underdiscussed second-order effect: comprehensive hreflang annotation increases the crawl coordination Google must perform, and on large estates that interacts with crawl budget.

Methodology: we analyzed server logs from two estates exceeding 200k localized URLs, before and after full hreflang deployment.

What the logs suggested:

— Validating an annotation requires crawling both ends of every pair. A cluster of 20 locales is 20 pages that must each be fetched and reconciled before any swap is trusted. At scale, that's a lot of coupled fetches.

— Estates with bloated near-duplicate locale sets effectively multiplied their crawl surface. Google spent cycles re-validating low-value alternates instead of discovering fresh content.

— Sitemap-based hreflang mitigated this somewhat by centralizing the relationship data, reducing per-page reconciliation crawls.

Nuance: this is a large-estate concern. Below a few tens of thousands of URLs, crawl budget is rarely the binding constraint and this effect is negligible.

Caveat: we can't directly observe Google's crawl-scheduling logic; we infer coupling from request patterns in logs, which is indirect.

Limitation: two estates is a thin sample for a budget claim — treat this as a hypothesis worth monitoring, not a settled result.

Conclusion: on very large estates, prune dead and near-duplicate locales, prefer sitemap-declared hreflang, and watch logs for re-validation churn crowding out discovery of new pages.