Gorilla stores a timestamp as a difference of differences, because the points of a time series arrive at roughly equal intervals.
A time series is one key, such as sales, and a series of points, each a timestamp and a value. Gorilla shards on the key, so a series is kept on a single machine, and it compresses both the timestamp and the value.
A server emits a point every five seconds, so the timestamps are roughly equidistant:
- The timestamps. The seconds are 10, 15, 21, 24, 29, 34 and 39.
- The differences. They are 5, 6, 3, 5, 5, 5.
- The differences of the differences. They are 5, 1, minus 3, 2, 0, 0. When every difference is 5, all are 0.
96% of timestamps are compressed to one bit, a zero. A value between 32 and minus 32 takes two bits, and between a thousand and minus a thousand, 10 bits. This gives 12x compression on timestamps.
Only the first timestamp is stored whole. The rest are differences, or differences of differences.