Case studies: noindex vs robots.txt, in the logs

Case studies: noindex vs robots.txt, in the logs

Which actually saved crawl budget? The log decided.

→ Distilled comparison — applied noindex to one thin section, robots.txt Disallow to another, then watched Googlebot. Robots.txt cut crawls to that path 96% in days; noindex pages kept getting crawled for weeks.
Takeaway: noindex de-indexes but doesn't stop crawling; robots.txt does the reverse.

⭐ Pick of the week — Onely combined-signal study — showed the bot must crawl a page to see its noindex, so blocking it in robots.txt can strand it in the index. Logs explained the contradiction.
Takeaway: never robots.txt-block a page you want de-indexed.

→ Google robots vs noindex doc — the canonical distinction.
Takeaway: pick the lever that matches your goal.