<b>How big a crawl sample you actually need</b>

<b>How big a crawl sample you actually need</b>

When you can't crawl 4M URLs, how many do you sample? For a site-wide error rate near 5%, to hit a ±1.0% margin at 95% confidence you need ~1,825 URLs — regardless of whether the site has 100k or 10M pages.

— ±3.0% margin: ~203 URLs ▓░░░░░░░░░
— ±2.0% margin: ~456 URLs ▓▓░░░░░░░░
— ±1.0% margin: ~1,825 URLs ▓▓▓▓▓▓▓░░░

Most auditors over-sample one template and under-sample the long tail, then report a biased rate.

So what: stratify by template, then random-sample within each. A 2,000-URL stratified sample beats a 200k unstratified crawl for estimating true health.