Playbook: join logs to a crawl for orphan + waste detection

Playbook: join logs to a crawl for orphan + waste detection
The cross-reference that exposes what crawlers see but you forgot about.
→ Screaming Frog's log-file analyzer import flow — drop logs + a crawl, it diffs them. Takeaway: "URLs in logs not in crawl" = orphans Google found via old links or external sites.
→ Sitebulb's log-vs-crawl docs — adds the inverse view, crawled-but-never-bot-hit. Takeaway: pages you care about that Google ignores.
★ Pick of the week — Kevin Indig's three-circle Venn — sitemap, crawl, logs as overlapping sets. Takeaway: each non-overlap region is a specific, nameable bug to fix.
→ pandas merge(how='outer', indicator=True) — roll your own join if you live in notebooks. Takeaway: _merge column labels each row's source set for free.
The diff, not either dataset alone, is where the action items live.