Mistake: trusting X-Forwarded-For for bot verification

Mistake: trusting X-Forwarded-For for bot verification

Client-supplied forwarding headers are spoofable, so verifying Googlebot off them lets fakes in. Sources:

→ OWASP / MDN on X-Forwarded-For — Anyone can set the header; only the IP your trusted proxy appended is reliable.
Takeaway: verify against the connection IP or your CDN's authenticated header, never the raw XFF.

→ Cloudflare docs (CF-Connecting-IP) — Use the provider's trusted header and lock origin to the CDN.
Takeaway: drop or distrust XFF chains from untrusted hops.

★ Pick of the week — a trusted-proxy parsing note — How to take the right-most untrusted IP from the chain.
Takeaway: pin your trusted-proxy list, then verify Googlebot on the real source.