Crawlable 'next month' links create infinite crawl

Crawlable 'next month' links create infinite crawl

Log review of 73 sites with date widgets: calendar 'next →' links generated a median 41,000 crawled URLs reaching years into the future, none with content.

— Sites with calendar/filter loops: 23.0% ▓▓░░░░░░░░
— Median junk URLs crawled: 41,000
— p90: 380,000
— Share of crawl on empty future pages: 22.0%

The trap: each empty future date is a fresh crawlable link to yet another empty future date — an infinite space that drains budget from real pages.

Fix: nofollow or remove the forward-paging links past a sane horizon, return 404 for empty dates, and block the date-param pattern in robots.txt. Cohort that capped the horizon cut crawled URLs by ↓ 89% and redirected budget to live content.

So what: any link that always generates one more link is a budget black hole.