<b>The crawl-to-index gap: joining logs with GSC</b>

<b>The crawl-to-index gap: joining logs with GSC</b>

The most useful analysis isn't logs alone — it's logs joined against what Google actually indexed. References.

→ Google's Index Coverage / Page Indexing report docs — your 'indexed' truth source to join against.
Takeaway: 'Crawled - currently not indexed' is the bucket to hunt.

⭐ Pick of the week: the URL-set-difference method many analysts use — list of URLs Googlebot hit (from logs) minus URLs in the indexed export = crawl waste.
Takeaway: <code>comm -23</code> on two sorted URL lists does it in one line.

→ Screaming Frog's guide to crossing log data with a crawl + GSC API pull.

→ OnCrawl's 'crawl ratio' concept — % of known pages Googlebot visited in a period.

Credit to Google, Screaming Frog, and OnCrawl.