April 2023: Distributed Consensus and Load Balancing

video locked
Enroll to watch

130 mins

The video discusses the issue of ordering events in a message queue or event bus with multiple topics, using examples of services posting response times and events like create, update, and delete. Enforcing order by timestamps or using distributed consensus algorithms like Paxos or Raft is mentioned as a solution. The use of logical clocks like hybrid logical clocks or Lamport clocks is also discussed as an alternative to the Spanner implementation, which requires Google-level hardware.

We also discuss the importance of enforcing order in the queue to ensure the sequential execution of instructions in a multi-threaded environment. It explains how a thread pool can mimic a queue and how events can be sent to different threads depending on whether they need to be sequentially ordered. The video also addresses questions related to hashing, partitioning, and handling multiple events across different services. Finally, it suggests that ordering across services can be complex and require engineering coordination.