The internal links that led Googlebot off a cliff

The internal links that led Googlebot off a cliff

A documentation site had deep, well-structured content but Google only ever indexed the top two levels. Anything three clicks deep stayed 'Discovered — currently not indexed.'

The navigation worked perfectly for users. Click through, pages load. So we crawled it the way a bot does — fetch HTML, follow hrefs, no JS execution of click handlers.

Their deep navigation links weren't real anchors. They were divs with onClick handlers that called the router programmatically. No href, nothing to crawl. The deeper pages existed and rendered, but the only path to them was a JavaScript click event Googlebot doesn't dispatch during discovery.

Googlebot renders JS, but it discovers URLs from <a href> attributes in the DOM — not from click handlers. A nav built on onClick is invisible to discovery even when it renders fine.

Fix: every navigation element became a real <a href> with the router intercepting clicks for the SPA experience. Best of both.

Result: deep pages indexed within four weeks, indexed page count up 3.4x, long-tail documentation traffic +90%.