Step one: prove you even have a crawl budget problem
Everyone jumps to fixes. Nobody checks if there's a problem. Here's the actual diagnostic, in order:
— Open Crawl Stats (Settings → Crawl stats in Search Console). Note total crawl requests/day.
— Divide your indexable URL count by daily crawl requests. That's roughly how many days to crawl the whole site once.
— If it's under ~10 days, stop. You don't have a budget problem.
— Check the "By response" chart. Mostly 200s? Fine. Bloated with 304/404/5xx? That's your real lead.
— Look at "Host status." Any red? That outranks every other concern.
Actually, most sites that run this test discover their crawling is fine and indexing is the bottleneck. Different problem, different fix.
Don't optimize a constraint you haven't measured. Run the test first.
Budget Myths
@CrawlBudgetMyths
Step one: prove you even have a crawl budget problem
Этот пост опубликован в Telegram-канале Budget Myths. Подписаться можно по ссылке: @CrawlBudgetMyths.