Gorilla is the time series database Facebook built.
In 2013 Facebook kept its time series data in HBase. A query from an engineer or an external system took a few seconds.
- Automations timed out, and the time a query took was erratic.
- Engineers stopped using the system. For a query such as new friends added in the last seven days, they cached the data in their own stores.
- Many engineers worked on the same problem. The effort was duplicated, and engineers were the most expensive resource.
- Facebook decided to build one general time series database for the queries of all engineers. Engineers adopt a system only when it is easy to use and easy to understand.
The existing system was 2 petabytes, and migrating it is a challenge.
OpenTSDB, Whisper and InfluxDB are more flexible, but they are on disk, and a response takes seconds.
Memcache is well tested at Facebook and is great for read-heavy systems. In some of Twitter's caches 60% of the queries are writes, and Memcache performs poorly there.