They noindexed 400K filter URLs. Traffic didn't move.
Everyone says faceted nav is eating your crawl budget. So a fashion retailer I looked at slapped noindex,follow on 412,000 color/size filter combos and waited for the floodgates.
Here's the thing: Googlebot still has to crawl a page to see the noindex tag. Log files three months later showed bot hits on those URLs up 6% — noindex doesn't reduce crawling, it requires it.
What actually moved the needle: a Disallow in robots.txt on the filter parameter. Crawl on those paths dropped 71% in two weeks. Indexed product pages got recrawled 2.4x more often.
No, the noindex tag is not a crawl-saving tool. It never was.
If you want Google to stop crawling, you block it. If you want it to stop indexing, you noindex. Different problems.
Budget Myths
@CrawlBudgetMyths
They noindexed 400K filter URLs. Traffic didn't move.
Этот пост опубликован в Telegram-канале Budget Myths. Подписаться можно по ссылке: @CrawlBudgetMyths.