> First of all remember that Paxos is a family of protocols for solving consensus... Raft on the other hand is a concrete protocol with well-defined, specified behavior. In fact, Raft is essentially an implementation of MultiPaxos... You have two choices: (1) You can use a library which provides a simple model (a log of commands), but doesn't scale well or (2) You can use a more complicated consensus algorithm and then deal with all of the Hard Problems™ that comes with it.
At AWS [0], everyone (I spoke to) who worked on distributed consensus had this exact same opinion, so you're not at all off the mark.
> A good example is CockroachDB: They're using plain Raft for writes, but uses "leader leases" for scaling reads.
The Chubby paper by Google [1] goes in to excruciating details of running a production Paxos system.
> Focus on how you can avoiding using a consensus algorithm due to the way your system works.
Amazon SQS may be one such example: I'd presume, it scales by avoiding consensus, in a way, simply maintaining multiple copies [2] and by placing guard-rails around delivery [3][4], ingestion, and duration of storage [5].
> First of all remember that Paxos is a family of protocols for solving consensus... Raft on the other hand is a concrete protocol with well-defined, specified behavior. In fact, Raft is essentially an implementation of MultiPaxos... You have two choices: (1) You can use a library which provides a simple model (a log of commands), but doesn't scale well or (2) You can use a more complicated consensus algorithm and then deal with all of the Hard Problems™ that comes with it.
At AWS [0], everyone (I spoke to) who worked on distributed consensus had this exact same opinion, so you're not at all off the mark.
> A good example is CockroachDB: They're using plain Raft for writes, but uses "leader leases" for scaling reads.
The Chubby paper by Google [1] goes in to excruciating details of running a production Paxos system.
> Focus on how you can avoiding using a consensus algorithm due to the way your system works.
Amazon SQS may be one such example: I'd presume, it scales by avoiding consensus, in a way, simply maintaining multiple copies [2] and by placing guard-rails around delivery [3][4], ingestion, and duration of storage [5].
[0] https://aws.amazon.com/builders-library/leader-election-in-d...
[1] https://blog.acolyer.org/2015/02/13/the-chubby-lock-service-...
[2] https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQS...
[3] https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQS...
[4] https://patents.google.com/patent/US10362131B1/en
[5] https://patents.google.com/patent/US8261286B1/en