Mistake: thinking noindex stops the crawling

Mistake: thinking noindex stops the crawling

People slap noindex on thin pages and assume Googlebot will stop visiting them. Then they're shocked the logs still show daily hits.

Actually, that's by design. To see a noindex tag, Google has to crawl the page first. And it keeps re-crawling — it can't know you didn't remove the tag unless it checks. Noindex is an indexing directive, not a crawling one.

If your real goal is to stop the crawl, robots.txt disallow is the lever. But careful — disallow blocks the crawl, so Google never sees the noindex, and a disallowed URL with external links can still get indexed URL-only.

Pick the actual problem you're solving. Index bloat? Noindex. Crawl waste? Disallow. They are not the same knob.