Caching is taking a temporary copy of the data and storing it close to what needs it, for low latency.
The call to the server for the list of t-shirts is slow. Each fix creates the next problem:
- An in-memory cache, a map of product ID to product data inside each server. It is fast and simple, but a restart loses it, and coordination between the caches is complex.
- A single global cache such as Redis. It holds one copy of the data and scales independently of the application servers. As the system scales, one cache server is not enough.
- Sharding. The hash of the t-shirt ID modulo the number of servers gives the cache. Going from 3 servers to 4 leaves 1 by 12 of the keys where they are.
- Consistent hashing. Server IDs are hashed onto a ring, and a request goes to the next clockwise server. Removing a server impacts only one server.
With few servers the chance of a skew is high. Plotting 20 virtual points for each server reduces it.