Budget Myths
Budget Myths
@CrawlBudgetMyths

An events calendar created infinite future URLs. Googlebot crawled to year 2147.

An events calendar created infinite future URLs. Googlebot crawled to year 2147.

Favorite real trap. A venue site's calendar widget had "next month" links with no end date. Every month was a unique crawlable URL: /events/2089/04, /events/2147/11, on and on.

Googlebot, being thorough, followed them. Logs showed it crawling event pages over a century into the future — tens of thousands of empty months — while real, current event pages waited days for a recrawl.

Fix took one line: nofollow on the "next" navigation past a 12-month horizon, plus blocking deep-future paths.

Within two weeks, crawl refocused on the present. Current events indexed same-day.

Here's the thing — your crawl budget wasn't small, it was being marched into the 22nd century.

Infinite navigation is the #1 self-inflicted crawl wound. Cap your calendars.
Этот пост опубликован в Telegram-канале Budget Myths. Подписаться можно по ссылке: @CrawlBudgetMyths.
tech

Свежие посты в категории «Tech Infrastructure»

Все каналы категории →

start

Готовы запустить рекламу через сеть public.tg?

Новый оффер, продукт, GEO, кейс, событие или партнёрский запуск — соберём маршрут под задачу и отдадим медиаплан.

Telegram для медиаплана: @AFFtop_connect. Быстрый тест: $20 за канал, $1000 за пакет по сети.