Memcached at Facebook sends a get request over UDP and a set or delete request over TCP.
TCP is acknowledgement based, so all writes are ordered. UDP is best effort and connectionless: no connection is created, and TCP connections are limited for a box and cost something. Facebook noticed a 20% reduction in request latency for get requests.
A write can then be processed after the read that follows it:
- Three requests are sent: get key 1, set the key to 30, get key 1.
- Handled in parallel, the two gets are processed before the set.
- Read after write consistency breaks. A user likes a post, refreshes the page, and the like is not there.
A smart client avoids this with a version number. The home page is version 29 and the like is version 30. A reply from the cache at version 27 is treated as an error on the client, which does not try to recover from it.
The client here is the one hosted on the web server, not the UI client.