<b>Noindex audits: the CMS defaults that silently nuke pages</b>

<b>Noindex audits: the CMS defaults that silently nuke pages</b>

Most 'why won't it index' tickets I get are a stray noindex nobody set on purpose.

How I run the audit:
— Crawl with Screaming Frog, filter Directives → 'Noindex', then cross-check against pages that <i>should</i> rank
— Check both meta robots AND the X-Robots-Tag HTTP header (people forget the header)
— Inspect staging→prod: 'Discourage search engines' in WordPress often ships to live

Usual culprits:
— Yoast/RankMath archive + tag settings
— Shopify's automatic noindex on filtered collection URLs
— A CDN/WAF rule injecting X-Robots-Tag

Where a crawler falls short:
— It won't catch JS-injected robots meta unless you render — enable rendering mode

Pros: noindex bugs are the fastest indexing win there is.
Cons: easy to miss the header-level ones.

Best for: any sudden deindexing — check this before anything else.

Not the place to look if pages were never indexed at all.