Server response time sets your crawl ceiling
Googlebot throttles to protect your origin. Across 41 log studies, median crawl rate vs. p90 server response time:
— TTFB ≤200ms: ~41 URLs/sec capacity ▓▓▓▓▓▓▓▓▓▓
— TTFB 200–500ms: ~22 URLs/sec ▓▓▓▓▓░░░░░
— TTFB 500ms–1s: ~9 URLs/sec ▓▓░░░░░░░░
— TTFB >1s: ~3 URLs/sec ▓░░░░░░░░░
A site that cut p90 response from 820ms to 240ms saw daily crawled-URLs rise +112% in 6 weeks — no other change.
So what: on large sites, server speed is a crawl-budget lever, not just a Core Web Vitals one. Fix the slow p90 templates first; they're capping discovery.
Crawl & Render
@CrawlAndRender
Server response time sets your crawl ceiling
Этот пост опубликован в Telegram-канале Crawl & Render. Подписаться можно по ссылке: @CrawlAndRender.