16 July 2026
A 5xx spike costs you crawl rate for weeks, not hours Googlebot reads sustained 5xx as "back off." In 17 incident post-mortems, a 6-hour 5xx episode (median 8% of responses) cut daily crawl volume tha…
15 July 2026
URL parameter math: how 5 filters become 100k URLs Faceted navigation multiplies, it doesn't add. Five filters with 4, 6, 3, 8 and 5 values each, in any combination and order, generate up to 4×6×3×8×5…
14 July 2026
Trust logs over GSC for crawl reality GSC's Crawl Stats samples and aggregates; raw server logs don't. Across 41 paired datasets, GSC under-reported total Googlebot hits by a median 11.0% and misattri…
13 July 2026
When Google ignores your canonical Google treats rel=canonical as a hint, not a directive. Across 96 large sites, share of declared canonicals that Google overrode: — median override rate: 8.7% ▓▓▓▓░░…
12 July 2026
Lastmod only works if you don't lie Google uses sitemap lastmod as a re-crawl hint — but only after it trusts the signal. Sites that touched lastmod on every URL nightly (auto-generated, not real edit…
11 July 2026
Server response time sets your crawl ceiling Googlebot throttles to protect your origin. Across 41 log studies, median crawl rate vs. p90 server response time: — TTFB ≤200ms: ~41 URLs/sec capacity ▓▓▓…
