<b>Server response over 600ms throttles crawl rate measurably</b>

<b>Server response over 600ms throttles crawl rate measurably</b>
We correlated median server response time against Googlebot requests-per-day on 183 sites.

— TTFB under 200ms: 1.0x crawl rate (baseline) ▓▓▓▓▓▓▓▓▓▓
— 200-600ms: 0.84x ▓▓▓▓▓▓▓▓░░
— 600ms-1.2s: 0.61x ▓▓▓▓▓▓░░░░
— Over 1.2s: 0.38x ▓▓▓▓░░░░░░

Googlebot's host-load logic backs off when your server slows — it's protecting your infrastructure, but it's spending your crawl budget at 38% efficiency. The drop is non-linear past 600ms.

During a load spike on one site, crawl rate fell ↓ 54% within 48 hours and took 9 days to recover after the fix.

So what: TTFB is a crawl-budget lever, not just a UX metric. Shaving 400ms off a slow backend can buy back 40%+ crawl capacity on large sites.