Rate Limits Are a Feature, Not a Bug

Most scraper "incidents" I'm pulled into start the same way: someone shows me a graph of 429 responses and asks how to make them go away. The honest answer — that nobody likes — is that the 429s are the well-behaved part of the system . The rest is what's broken. I'm going to argue that rate limits are not your enemy. They're a contract. And scrapers that treat them like a contract — instead of an obstacle — are the only ones I trust to run unsupervised for more than a quarter. The teardown Thre