Mistake: shipping faceted URLs to the crawler before deciding which combos earn an index slot

Mistake: shipping faceted URLs to the crawler before deciding which combos earn an index slot

Teams launch a /category/{a}/{b}/{c} template, link every filter combination, and discover 400k crawlable URLs from 12k products. Crawl budget evaporates on combinations nobody searches.

The combinatorial guardrail:
— Step 1. Owner: SEO. Define the allow-list: which facet dimensions and which depth (usually 1, occasionally 2) are indexable. Everything else is noindex + nofollow on the link.
— Step 2. Owner: dev. Render indexable combos as crawlable ; render non-allow-listed combos behind a button or with rel=nofollow and a noindex header.
— Step 3. Owner: SEO. Confirm each allow-listed combo has documented search demand. Gate: no combo ships without a demand number attached.
— Step 4. Owner: dev. Log generated URL count per deploy. Gate: count within 10 percent of the planned figure, else rollback.

Guardrail: never let the link graph generate URLs faster than you can justify them.

Ship gate: don't publish until all boxes are checked.