<b>The faceted noindex trap that killed 40% of a client's crawl budget</b>
Client slapped <code>noindex,follow</code> on every filter URL. Felt smart. Wasn't.
Googlebot still crawled all 11,000 facet combos to read the noindex tag. Crawl budget gone, money pages crawled once a month.
Here's the move:
— Pick 5-8 high-volume filter combos people actually search ("red running shoes size 10"). Make those real, indexable, linked-to pages.
— Everything else: block the parameter in robots.txt so the bot never fetches it.
— Kill the <code>follow</code> fantasy. Noindex,follow stops passing signal after ~6 months anyway.
Watch your internal links when you do this. If a blocked facet was your only path to a product, you just orphaned it. Map paths first, block second.
Sitemap Hustle
@SitemapHustle
<b>The faceted noindex trap that killed 40% of a client's crawl budget</b>
Этот пост опубликован в Telegram-канале Sitemap Hustle. Подписаться можно по ссылке: @SitemapHustle.