The noindex-then-block sequence everyone gets backwards
Want a page gone from the index AND off Google's crawl list? Order matters, and most people invert it:
— Add noindex to the page (meta robots or X-Robots-Tag header).
— Leave it crawlable. Google must crawl it to SEE the noindex.
— Wait until it actually drops from the index — check with the URL inspection tool, not your gut.
— Only THEN, if you want to save crawl, add a robots.txt Disallow.
Do it in reverse and the page stays indexed forever — Google can't read the noindex it can't fetch.
Actually this is the single most common self-inflicted wound in this niche: blocking a page to remove it, which guarantees it can't be removed.
Crawl first, noindex applies, THEN block. Any other order is a bug.
Budget Myths
@CrawlBudgetMyths
The noindex-then-block sequence everyone gets backwards
Этот пост опубликован в Telegram-канале Budget Myths. Подписаться можно по ссылке: @CrawlBudgetMyths.