Mnesia new feature for eventual consistency with CRDT

Hi,

I am thinking about adding a new eventual consistency model for Mnesia. In particular, this is trying to address the question that when we are doing dirty async operations on Mnesia, according to the doc: “notice that there is a risk that the database can be left in an inconsistent state if dirty operations are used to update it”, and from mnesia documentation " Determining what data to keep after a communication failure is outside the scope of Mnesia". Both of these problems can be solved to some extend with CRDT (conflict-free replicated data types), and this has already been implemented by databases like Riak and Redis.

I wanted to start by treating each mnesia table as, say a set (we can think about bag, etc later on). And this can be a G-set or some other similar type of CRDT set. In this way we allow two replicas to converge even when they receive asynchronous operations. Later on, we can do more fine grained resolution such as allowing a filed of a record in a mnesia table to be a CRDT, etc.

I will start working on this on a fork of the otp repo, and then if this is goes well, consider PR it to the main repo. If anyone has any feedback/suggestions, please do let me know.

Thanks!

4 Likes