Roundup: log spikes that weren't Googlebot

Roundup: log spikes that weren't Googlebot

Traffic anomalies, diagnosed from the access log.

→ Cloudflare incident writeup — a 20x request spike looked like a crawl surge; user-agent analysis showed 92% was a single scraper rotating IPs and faking Bingbot. Rate-limiting restored origin headroom.
Takeaway: always confirm the bot before you celebrate or panic.

⭐ Pick of the week — practitioner AI-crawler study (Aleyda Solis roundup) — quantified the new wave: GPTBot, ClaudeBot and others made up 28% of bot requests in one log sample, none of it search traffic. Decisions on whether to allow them got data-driven.
Takeaway: AI crawlers are now a measurable line item in your logs.

→ Dark Visitors AI-agent list — the reference for identifying every new bot UA.
Takeaway: keep a current UA dictionary handy.