Full crawl vs. stratified sample: the audit-time tradeoff

Full crawl vs. stratified sample: the audit-time tradeoff

On sites over 1M URLs, a full crawl is often theater. Compared full vs. stratified-sample audits on 44 large domains for issue-detection accuracy.

— Full crawl: 100% coverage, median 31h run, $$$ ▓▓▓▓▓▓▓▓▓▓
— Stratified sample (n=2,400, by template): caught 96.8% of distinct issue types, 1.4h run ▓▓▓▓▓▓▓▓▓░

The 3.2% missed were one-off URLs, not systemic patterns.

So what: full crawls find every instance; samples find every pattern — and patterns are what you fix. For templated sites, a stratified sample (≥384 URLs per template for ±5% confidence) surfaces nearly all systemic issues at ~5% of the runtime. Reserve full crawls for sizing remediation scope after the sample tells you what's broken.