A chat widget was eating the crawl budget of an entire site

A chat widget was eating the crawl budget of an entire site

A B2B software company couldn't get new pages indexed. They'd publish, wait three weeks, and the URLs stayed in "Discovered — currently not indexed." The site was small, fast on paper, fully SSR. Nothing explained it.

We ran a crawl with a JS-rendering crawler and watched the network panel. Every page pulled a 740KB customer-chat bundle, render-blocking, from a third-party domain that frequently took 8–12 seconds to respond. Googlebot, rendering each page, sat waiting on that script before it would consider the DOM settled.

With a tight per-host render budget, the crawler was burning almost its entire allowance on one slow vendor script — leaving nothing to discover and fetch new URLs.

We deferred the widget behind a click and added rel=preconnect for the rare loads. Googlebot stopped waiting.

—New-page indexing time: 21 days → 2 days
—Pages crawled per day (server logs): 180 → 1,150

The content was perfect. A vendor's blocking script was strangling the crawler before it could read any of it.