A flash sale sells the t-shirts at a 50% discount for two hours, and the system survives the spike by dropping the requests it cannot process.
The founder expects 10x the daily sales. The preparation has three steps: estimate the number of requests, preemptively scale the servers, and cache along the access patterns. Load testing finds how many requests one server can serve.
The sale starts with 1 million concurrent users and the system collapses. The logs show the same user IDs making thousands of requests in seconds: a distributed denial of service attack.
The fixes for the second sale, a week later:
- A web application firewall blocks bots at the CDN and the gateway.
- Graceful degradation. Payments, customer support and order status get the highest request priority. Short circuit fails requests in the upstream service, and backpressure drops them on arrival downstream.
- Concurrency with multiple threads, and gRPC, remove head of line blocking.
At extreme scale the answer is rate limiting: a server that can manage 100 requests and gets 1000 drops 900.