Playbook: contain a faceted-nav crawl explosion

Playbook: contain a faceted-nav crawl explosion
Uncontrolled facets generate a median 47x more crawlable URLs than products (sample: 38 e-commerce crawls). Worst case observed: 410x.

Containment sequence:
— Step 1: enumerate param combinations crawled in logs; rank by hit volume
— Step 2: classify each param: indexable-value (color, brand) vs. noise (sort, sessionid, page beyond 1)
— Step 3: noise params get robots.txt disallow + canonical to clean URL. Typical kill: 78% of facet URLs ▓▓▓▓▓▓▓░░░
— Step 4: indexable facets — allow ONLY single-param combos, block multi-select stacking
— Step 5: re-measure crawl-URL count at 30d; target 90%+ reduction in facet hits

So what: never block by guesswork. Rank params by actual crawl hits first — two or three noise params usually cause 80% of the explosion.