Mcsqueal reads the binary log of a MySQL database and broadcasts every committed update to the other databases and to the caches that use them.
Adding servers the naive way lets every web server connect to every memcache server. Facebook broke its architecture into clusters instead. The web servers of a cluster talk only to its own memcache servers, and each cluster is responsible for the entire data set.
Each cluster has its own database, so the databases are not consistent. A web server that broadcast every set operation to all databases would be too expensive. The log does the work:
- MySQL notes every operation down in its binary log, which it replays after a collapse.
- One of five databases commits an update to its log.
- After some time mcsqueal reads the log and broadcasts the update to the other four.
- One line before each update, mcsqueal has written a delete of that key from memcached.
- The relevant caches delete the key.
A single update reaches every database and cache, which gives reasonably high consistency.