A request that reads many keys at once is served better by replicas of a cache than by its shards.
One home page request fans out into a get for keys 1 to 100. Three cache servers answer it slowly, and there are two ways to scale them:
- Sharding. The servers take keys 1 to 33, 34 to 66 and 67 to 100. The get is split into three pieces, then six when the shards are cut again. With a thousand users every shard still gets a thousand requests, so each has to scale from 100 to a thousand. The problem is not solved.
- Replication. Each server holds all keys from 1 to 100. One get goes to a single server, the next to another.
Sharding is far better when there is no aggregation, because a request is routed to the right place.
A cache that is out of memory evicts keys. A database cannot, so it has to shard or replicate. In the worst case a cache reads from the database, so durability does not matter.