Today, I remembered an (internal) component called WANDist that I have seen mentioned in various Whatsapp presentations. Its purpose is to connect disjoint Erlang clusters running in different data centers. (over TLS, obviously)
Do you know of anything similar to WANDist and open-source out there?
We have deprecated and removed wandist. It was slower, consumed more CPU & RAM, and suffered from head-of-the-line contention. Erlang distribution (especially with kTLS enabled) is a better alternative.
With pg it is possible to create any number of disjoint sub-clusters (by running several pg scopes). So we are essentially back to .hosts.erlang, but instead of global (which we disable), we use our own gen_server that simply connects to the list of nodes in that file.
Yet to answer your question, you might want to look at Partisan. It does not seem to be maintained though.
Thank you for your insights & information!
I’ve played with Partisan a (rather long) while back. While Lasp-lang is archived, Partisan is probably still a viable approach for a membership system, using different gossip protocols.
I’ll look into your suggestions for pg etc.