"Block low-value pages in robots.txt to remove them from Google."

"Block low-value pages in robots.txt to remove them from Google."

That does the opposite of what you think.

A site wanted 12,000 thin tag-archive pages out of the index. They added Disallow in robots.txt and waited. Three months later the pages were still indexed — showing in results with "No information is available for this page." Worse than before.

The reason is a classic on-page trap: robots.txt blocks crawling, not indexing. Google couldn't crawl the page to see the noindex tag it would've otherwise obeyed, so it kept the URL in the index based on external links. They'd locked the door on their own removal instruction.

Fix: they removed the Disallow, added <meta name="robots" content="noindex">, let Google crawl and read it. Within six weeks, 11,300 of the 12,000 dropped out. Index count fell, relevant pages firmed up.

To deindex, you must let Google in to see the noindex. Blocking the crawl traps the page in limbo. (The fix was unlocking the door, not bolting it harder.)