Borg is the system at Google that provides compute and memory for processes and manages each one's life cycle from start to finish.
Borg finds the data centers, allocates resources and restarts a process that crashes. 98% of the servers at Google run because of it.
- A job's priority is monitoring, production, batch or free. A higher priority job preempts a lower one. Production jobs cannot be preempted, which avoids a cascade.
- A job runs in a Linux container, not a virtual machine. A chroot jail manages security and a cgroup manages resources.
- A cell is about ten thousand machines under one Borg master, which places a task by a hybrid of best fit and quick assignment.
- A running task gets a mapping in BNS, the Borg naming service. Chubby stores it in a file with its state and health.
- The Borg master has five in-memory replicas with re-election using Paxos. If it dies, the Borglets continue functioning.
Sending a task where its libraries are installed reduces the start time from 25 seconds to five.