Edge rate limit vs. WAF signature: a Lagos API blocks scrapers
A pricing-data API got scraped hard. The team first wrote a WAF rule matching the scraper's user-agent string. The scraper rotated user-agents every hour, and within a day the signature was useless.
Scraped requests still pulled 1.1M hits/day off a metered origin.
They switched tactics: an edge rate limit keyed on the combination of IP plus requested endpoint plus an absence of a valid API token. Behavior, not identity.
Scrapers couldn't rotate their way past a per-key request budget. Unauthorized hits to origin dropped 89%. Paying customers with tokens never tripped the limit.
WAF signatures win against known exploit patterns — SQL injection, specific CVEs. Behavioral rate limits win against adaptive abuse that changes its fingerprint faster than you can write rules.
The number that mattered: 89% fewer scraper hits at origin.
Edge of Glory
@EdgeOfGloryCDN
Edge rate limit vs. WAF signature: a Lagos API blocks scrapers
Этот пост опубликован в Telegram-канале Edge of Glory. Подписаться можно по ссылке: @EdgeOfGloryCDN.