Deciding if you're actually a "big site" — the threshold checklist

Deciding if you're actually a "big site" — the threshold checklist

Crawl budget is explicitly a large-site topic. Before you spend a week on it, classify yourself honestly:

— Count your indexable, unique, valuable URLs (not tag pages, not params).
— Under ~10,000? Google's own guidance says you basically don't need to worry about crawl budget. Stop here.
— 10k–1M with frequent change? Now it's worth attention.
— Over 1M, or 10k+ that change daily? This is the actual audience for crawl-budget work.
— Recheck after pruning junk — most "million-URL" sites are 50k real pages plus 950k filter combinations.

Actually Google literally names those thresholds in its large-site crawling docs. If you're below them, you're optimizing a non-problem.

Crawl budget is a big-site problem. Run the count before you assume you have a big site.