Zanzibar caches the results of hot keys on both the caller and the callee, and collapses identical requests that are in flight into one.
A check on the same group repeats its lookups, so the members, the admins and the tuple set are each cached. Cache keys have timestamps in them. The evaluation timestamp, or zookie, is rounded to the next near value, which lets most requests use a cached result.
Request collapsing protects the database from a viral key:
- The first request for a key goes to the database, and the key is stored locally with a dummy value.
- A later request for that key finds the dummy value and waits.
- When the first query returns, every waiting caller is answered together.
A caller cancels the branches that can no longer change the answer, and the servers delete them from their queues. A second caller collapsed onto a cancelled request is left hanging. So a server keeps a count of the callers interested in a request, and does not cancel while more than one is waiting.