Index bloat prevention checklist
Run before opening the crawl floodgates on a scaled set.
☐ Step 1 (SEO): Define the minimum-viable-page threshold (data completeness + word count). Only pages above it enter the sitemap.
☐ Step 2 (Dev): Below-threshold pages render noindex,follow server-side, not via JavaScript. Gate: view-source shows the tag.
☐ Step 3 (SEO): Faceted/filter/sort/pagination combinations are excluded from the sitemap and canonicalized to the base view.
☐ Step 4 (Dev): Empty-result pages (zero rows) return the page but with noindex. Gate: never index a 'no results' template.
☐ Step 5 (SEO): Sitemap holds only canonical, indexable, 200-status URLs. Gate: validate with a status check before submit.
☐ Step 6 (Ops): Set a coverage alarm — if indexed count exceeds sitemap count by >15%, investigate parameter leakage.
Guardrail: index count should track sitemap count, not exceed it.
Ship gate: don't publish until all boxes are checked.
Scale Engine SOP
@ScaleEngineSOP
Index bloat prevention checklist
Этот пост опубликован в Telegram-канале Scale Engine SOP. Подписаться можно по ссылке: @ScaleEngineSOP.