The crawl rate limiter you used to rely on is gone
Old-timers still reference the Search Console crawl rate setting like it's there. Google retired it. The manual limit slider was removed because Googlebot now self-regulates from your server's responses.
What replaced it: speak in HTTP. A 500 or 503 tells Google to back off. Sustained fast 200s tell it the runway is clear. There's an emergency form for genuine overload, but day to day, your server's behavior is the rate control.
This flips the old advice. You don't ask Google to crawl less — you make your server look stressed (or healthy) and it adapts within minutes to hours.
The dial moved from the dashboard to your response codes.
Your server is the throttle now.
Budget Myths
@CrawlBudgetMyths
The crawl rate limiter you used to rely on is gone
Этот пост опубликован в Telegram-канале Budget Myths. Подписаться можно по ссылке: @CrawlBudgetMyths.