Dremel is used by product managers, operations teams, engineers, support teams and analysts, who expect answers in tens of seconds.
It is much faster than MapReduce, and the two differ in three more ways:
- The program. MapReduce runs any kind of program, and it is code. An engineer writes it. An analyst finds it difficult.
- The data. MapReduce parses unstructured data. Dremel needs structured columnar data.
- The work. MapReduce is good for joins across tables, such as Google Photos with Google Maps, and a join in code takes any arbitrary condition. Dremel is for aggregation over a column.
Dremel depends on Colossus for storage, Jupiter for the network and Borg, the cluster manager.
Dremel is not a database in itself. It is an analytics engine. Google keeps storage in Colossus, compute in Borg, configuration in Spanner and the network in Jupiter, and builds new systems on top of these systems. Monarch used Spanner for configuration and Colossus for storage.