Logfile Roundup
Logfile Roundup
@LogfileRoundup

Playbook: classify every bot in your logs

Playbook: classify every bot in your logs
Building a clean bot/human/fake-bot split, from sources that maintain the lists.
→ The ua-parser regexes (BrowserStack-maintained) — battle-tested user-agent patterns. Takeaway: don't hand-roll UA regex, this corpus already covers the long tail.
→ Matomo's bot detection list — open, frequently updated, MIT-licensed. Takeaway: a ready table of known crawler UAs to left-join against.
★ Pick of the week — Cloudflare Radar's verified-bots list — bots that publish IP ranges for verification. Takeaway: split your traffic into verified-bot / claims-to-be-bot / human, and treat the middle bucket as suspect.
→ Google's crawler overview page — distinguishes Googlebot, AdsBot, GoogleOther, inspection tools. Takeaway: tag each by purpose, not as one blob.
Three buckets beat two — the "claims-to-be" group is where the interesting story usually lives.
Этот пост опубликован в Telegram-канале Logfile Roundup. Подписаться можно по ссылке: @LogfileRoundup.
tech

Свежие посты в категории «Tech Infrastructure»

Все каналы категории →

start

Готовы запустить рекламу через сеть public.tg?

Новый оффер, продукт, GEO, кейс, событие или партнёрский запуск — соберём маршрут под задачу и отдадим медиаплан.

Telegram для медиаплана: @AFFtop_connect. Быстрый тест: $20 за канал, $1000 за пакет по сети.