Mistake: assuming noindex stops the fetches in your logs

Mistake: assuming noindex stops the fetches in your logs

Noindex pages still get crawled — sometimes more — and people miss it because they only watch the index. Reads:

→ Google Search Central (noindex) — Google must fetch the page to see the noindex tag, so it keeps crawling it.
Takeaway: logs are the only place you'll see that wasted budget.

→ Search Central (robots.txt vs noindex) — Blocking in robots.txt prevents the fetch but then the tag is never read.
Takeaway: choose the right tool — don't stack both and get neither effect.

★ Pick of the week — a log + noindex cross-check script — Joins crawled URLs against your noindex set.
Takeaway: heavily crawled noindex URLs are prime candidates to prune from internal links.