APIs and web pages need different rate limits, and here's why

APIs and web pages need different rate limits, and here's why

A browser loading a page makes a burst of requests, then goes quiet for the time it takes a person to read. An API client makes steady, evenly spaced calls, sometimes for hours.

The same limit applied to both fails in both directions: too tight for the page burst, too loose for a scraper politely pulling your catalogue at a steady rate.

For web pages:
— count only HTML responses or navigation requests;
— allow bursts, e.g. 60 per minute, since one visit may need 5–10 quickly;
— use a challenge, not a block.

For APIs:
— key on the API token or authorization header, not just IP;
— set per-endpoint limits, tighter on write operations;
— return a proper 429 with a Retry-After header so well-behaved clients slow down on their own;
— block, since a challenge page is meaningless to a machine.

Two rule sets, two philosophies. Trying to merge them is how you get both complaints and scrapers.