Managing Complex Distributed Applications?

Reading through Distributed Erlang and other resources, Erlang makes it easy to connect nodes and pass messages between them.

But taking a step back I’m still unsure what I should use to create/manage a distributed application. I would like to:

  1. Run a service on less-than-all nodes in a cluster while still being available to all nodes in a cluster through messages.
  2. Be able to call a service by name without needing to know before-hand which node/pid that service will belong to.
    • Possibly being able to run multiple of the same service on a node (in-node load balancing).
  3. Scale in/out a service either by adding nodes or electing existing nodes as servers of that service
  4. Be able to distribute requests for that service across nodes (cross-node load balancing).
    • By either latency, number of processes, or some combination.

Lets say user_serv is on 111.111.101,
pay_serv is on 111.111.102,
and inv_serv is on both 111.111.101 and 111.111.103.

user_serv wants to purchase an item so it needs to use the pay_serv. The pay_serv needs to fetch details for that item so it needs to use the inv_serv. These services do not know where the others are, so they have to use the reg_serv to find a node to talk to.


I think I could come up with something like the above where I could add nodes to a cluster just by connecting it to one node and letting gossip between reg_serv do the rest. But that is a lot of custom boilerplate that- while fun to think about- is probably just reinventing the wheel.

What are people using to create/manage complex distributed applications? Is everything I want already part of OTP and I missed it or are there frameworks for this sort of thing?


I guess what you are looking for is global or gproc? Both of which can be used with the built-in name registration facilities in gen_server and friends.


And if you’re after some redundancy/availability/load balancing, you might be interested in pg.
At the moment there is no out-of-the-box load balancer, but it’s really simple to build one on pg.