Mistake: gzipped sitemap served with the wrong headers

Mistake: gzipped sitemap served with the wrong headers

A sitemap.xml.gz served as text/html or with double-compression (Content-Encoding: gzip on an already-.gz file) arrives as garbage. The crawler downloads bytes it cannot parse.

Fix checklist:
— ✅ .xml.gz file served with Content-Type: application/x-gzip (or application/gzip)
— ✅ Do NOT also set Content-Encoding: gzip on a pre-gzipped file — that double-encodes
— ✅ Plain .xml served as application/xml or text/xml, never text/html
— Confirm the file decompresses to under 50MB uncompressed

Definition of done: curl -I the sitemap URL and read the headers, then curl ... | gunzip | xmllint --noout -. Headers correct + clean decompress + valid XML = pass. Any parse error means an encoding mismatch.