robots.txt Disallow ≠ noindex — and the trap that follows

robots.txt Disallow ≠ noindex — and the trap that follows

The single most expensive misunderstanding in indexing. Worth saying plainly.

The mechanics:
— Disallow blocks crawling, not indexing — a blocked URL with inbound links can STILL appear in results as a bare link ('Indexed, though blocked by robots.txt')
— noindex requires crawling to be SEEN — so if you Disallow a page, Google never reads its noindex tag

Testing tools:
— GSC robots.txt Tester (legacy) and Inspection both show the blocked state
— Inspection's 'Indexed, though blocked' warning is the tell

Where they fall short:
— No tool flags the contradiction of Disallow + noindex together — you have to know it's wrong

Pros: easy to verify the block in Inspection
Cons: tools confirm symptoms, not the logic error

Best for: understanding before you touch either file.
NOT for: removing pages — to truly de-index, allow crawl + noindex, OR use the Removals tool first.