The infinite calendar that crawled you into the ground

The infinite calendar that crawled you into the ground

Classic trap: an events or booking widget with "next month" links and no end. Googlebot follows December into next year, into 2189, into the heat death of the universe. Each click a new URL, each URL a crawl.

These spider traps generate unbounded URL spaces from a single template. Session IDs in URLs do the same thing — every visit a fresh path.

The fix: cut the loop. nofollow won't reliably save you here — disallow the calendar path in robots.txt, or stop generating navigable links past a sane horizon. Kill session IDs in URLs entirely.

Google can usually pattern-detect these now, but "usually" is doing heavy lifting on a slow site.

Don't build a maze and then complain the crawler got lost.