Memcached at Facebook is a common cache that answers the hundreds of key value reads behind a single page.
A home page is a tree of posts, comments and replies. A cache used by all engineers needs high adoption, so engineers import a client library, which compresses values and batches requests.
It is a look-aside cache. A get that misses goes to MySQL and populates the cache. A set updates the database and deletes the key, because a delete is idempotent.
The cache is scaled according to the access pattern:
- Protocols. Set requests use TCP, so they go in order. Get requests use UDP, which saves 20% in request latency.
- Replication. Sharding spreads the keys of a page over all nodes. With replication, one node answers the whole page.
- Rate limiting. Each client keeps a sliding window that is neither too large nor too small.
- The gutter. Spare instances, around 1% of all nodes, reduce client failures by 99%.
- Mcsqueal. The keys to delete are added to the MySQL binary log, which is propagated.