Status-code mining: three logged results

Status-code mining: three logged results

What people found by grepping the response-code column.

→ Sitebulb soft-404 case — filtered logs for URLs returning 200 but flagged as soft-404 in Search Console; 4,100 thin pages were eating 22% of crawl. Returning a real 404 freed that budget within a month.
Takeaway: a 200 can lie; the log plus GSC together don't.

⭐ Pick of the week — Lumar (ex-DeepCrawl) 5xx hunt — spotted a recurring 503 spike every night at 02:00 in the logs, traced to a backup job. Fixing the schedule lifted successful crawls 14%.
Takeaway: timestamp patterns expose ops problems SEO tools miss.

→ JR Oakes status-code awk snippet — one-liner that tallies codes by directory. Credited and reusable.
Takeaway: tabulate by path, not just sitewide.