Budget Myths
Budget Myths
@CrawlBudgetMyths

You disallowed the page AND added noindex. Now neither works.

You disallowed the page AND added noindex. Now neither works.

The contradiction nobody catches: a URL is blocked in robots.txt, and you also put a noindex meta tag on it expecting it to drop from the index.

Here's the thing — Google can't read the noindex if it can't crawl the page. The disallow stops the fetch, so the meta tag is never seen. If that URL has inbound links, Google can index it URL-only, with no title, forever, and you can't get rid of it.

The fix: to deindex, you must allow the crawl so Google reads the noindex. Let it in, let it see the tag, let it drop the page. Then block later if you want.

Google's documentation states this directly. Two blockers cancel out. Pick one.
Этот пост опубликован в Telegram-канале Budget Myths. Подписаться можно по ссылке: @CrawlBudgetMyths.
tech

Свежие посты в категории «Tech Infrastructure»

Все каналы категории →

start

Готовы запустить рекламу через сеть public.tg?

Новый оффер, продукт, GEO, кейс, событие или партнёрский запуск — соберём маршрут под задачу и отдадим медиаплан.

Telegram для медиаплана: @AFFtop_connect. Быстрый тест: $20 за канал, $1000 за пакет по сети.