<b>Noindex audits: Screaming Frog vs the cheap mistakes</b>
A stray noindex is the most common 'why won't this index' cause I find. The crawler hunts them fast.
Setup: crawl, then Directives tab → filter Noindex. Two minutes on a 10k site.
What it does well:
— Catches meta-robots AND X-Robots-Tag HTTP headers — most people forget the header version entirely
— Flags noindex pages still sitting in your sitemap (a contradiction Google notices)
— Free tier covers 500 URLs, plenty for small sites
Where it falls short:
— Default crawl uses the raw HTML — JS-injected noindex tags need 'JavaScript rendering' enabled, which is slower and easy to forget
— Paid license is £199/yr once you outgrow 500 URLs
Pros: thorough, finds header-level noindex, exportable
Cons: misses JS noindex by default, paywall past 500
Best for: a full directive sweep before launch.
NOT for: a quick one-URL check — Inspection is faster.
Index or Bust
@IndexOrBust
<b>Noindex audits: Screaming Frog vs the cheap mistakes</b>
Этот пост опубликован в Telegram-канале Index or Bust. Подписаться можно по ссылке: @IndexOrBust.