Playbook: measure crawl frequency per site section

Playbook: measure crawl frequency per site section
Four sources for answering "how often does Google hit my product pages vs blog?"
→ OnCrawl's crawl-frequency segmentation — map URL prefixes to sections, then count bot hits per section per day. Takeaway: /product/ starved while /tag/ gorges is the classic red flag.
→ Logz.io's awk-by-path snippet — cut the path's first segment and uniq -c it. Takeaway: a 30-second view of where attention goes.
★ Pick of the week — Dawn Anderson's "crawl ratio" framing — bot hits divided by URL count per section. Takeaway: a low ratio on money pages is your real problem, not raw hit volume.
→ Google's crawl-stats report docs — cross-check your log numbers against Search Console's host breakdown. Takeaway: if they diverge, suspect a logging gap or a CDN swallowing hits.
Section-level is where crawl problems become visible — site-wide totals hide them.