A viral post is a single popular object that may have thousands of checks by different users.
Zanzibar is indexed on object ID, so a viral post creates a hot spot. Zanzibar detects popular objects by tracking the reads on an object. If the reads are greater than a threshold, all the relations of this object are cached. When the next user comes, his relation is already cached.
Rate limiting works in three places:
- Each client has a maximum of CPU seconds to be used per second. If the CPU seconds are greater than the threshold, as when Photos asks for a lot of relations, the client is rate limited.
- Each server has the same logic for outstanding RPCs.
- Requests to Spanner are rate limited for every client and every object and client pair. Different lock tables are used for different clients, so Spanner does not throttle them all.
Zanzibar holds 2 trillion relation tuples in 30 regions and answers 10 million queries per second, 99% of them reads. Callee caches save 12% of the reads.