Mistake: analyzing sampled CDN logs as if they were complete

Mistake: analyzing sampled CDN logs as if they were complete

Many CDNs ship a fraction of requests by default, and bot counts come out wildly low. Curated guidance:

→ Cloudflare Logpush docs — Logpush is unsampled; the dashboard Analytics are sampled. People grab the wrong one.
Takeaway: for crawl analysis pull Logpush (or Logpull) raw, never the sampled UI export.

→ Fastly real-time logging docs — You choose the sample rate in VCL; a stray 0.1 means you see 10% of Googlebot.
Takeaway: audit your logging VCL before trusting any hit counts.

★ Pick of the week — AWS CloudFront access logs note — Standard logs are best-effort and can be delayed or dropped; real-time logs are sampled per your config.
Takeaway: reconcile log volume against request counts so you know your true coverage.