Khepri consistency guarantees

[luos]

Hi,

I was wondering if Khepri provides consistency guarantees in the following (RabbitMQ), hypothetical case:

1. I create a binding between an exchange and a queue
2. Publish a message with that binding so that the message should go into a queue (same Channel process). I assume here that RabbitMQ will perform a "local" read to make sure that routing is fast.

In a case where the local Ra machine is behind (lets say the machine is very overloaded) it's majority the command may be committed - while not visible on the local node. And I was wondering if this case will be handled by Khepri - to for example make sure the local node is always caught up to the last command when the TX returns, or similar behaviour?

This is just a thought experiment - I am not 100% how RabbitMQ handles this at this time, though I can see it does ETS queries for the direct exchange for example.

[michaelklishin]

Local writes will be observable by reads on the same nodes.

[michaelklishin]

@luos the local node will always have its state machine (which for Khepri means its entire data set) updated after a majority of replicas responds. Leader node also cannot not be included into this majority. Therefore both node-local and leader queries will observe the most recent write.

[kjnilsson]

khepri will ensure that the local projections (ETS) are updated before a write operation returns. In fact the reply for a command is always delivered by the local node, not the leader if the leader is on a remote node.

Other non-local nodes may not have updated their projections by the time the command completes.

[luos]

Thank you for the answers, that makes sense.

Other non-local nodes may not have updated their projections by the time the command completes.

Do you think there is something which could be done, or is there something planned to make sure that at least all connected nodes are up-to-date, similarly how an mnesia tx would work? To minimise the chance of race conditions, for example when using multiple connections through a load balancer or similar.

[michaelklishin]

@luos Raft considers a write to be successful when a majority responds. Mnesia considers it to be a success when all replicas respond. There are no plans to build a copy of Mnesia.