What is a "distribution channel"?

Questions / Help
,
1

I’m trying to learn more about how the Erlang distribution works under the hood, and have been poking at busy_dist_port failures in particular. This paragraph in the erl_cmd docs intrigued me (emphasis mine)

A larger buffer limit allows processes to buffer more outgoing messages over the distribution. When the buffer limit has been reached, sending processes will be suspended until the buffer size has shrunk. The buffer limit is per distribution channel . A higher limit gives lower latency and higher throughput at the expense of higher memory use.

I cannot seem to find what this “distribution channel” is, or how it relates to the distribution buffer.

Some experimentation has shown me that I can deliberately fill the distribution buffer when sending messages to one node, and get suspended, but still end messages to another node. I assume from these experiments that a “distribution channel” refers to my connection to other nodes, but I was wondering if there was a more formal definition that I am not finding.

Is the “distribution channel” as simple as the connection between nodes? Is there a buffer on each end of the channel (on the sender and the receiver)?

If there’s somewhere I can read more about this, it would be much appreciated :slight_smile:

1 Like
2

Yes, the channel is sort of the connection between a pair of nodes. I imagine that the reason it is not called the connection is because, as Erlang allows alternative distribution carriers (that is you can implement your own), it is possible for one channel between two nodes to consist of multiple connections.

I good place to start reading more might be here: How to Implement an Alternative Carrier for the Erlang Distribution — erts v15.2.6

1 Like
3

Aha! This paragraph helps shed some light on it too

The soft busy flag implies that the driver can handle calls to the output and outputv callbacks although it has marked itself as busy. This has always been a requirement on drivers used by the distribution, but no capability information has been available about this previously. For more information, see erl_driver:set_busy_port()).

I guess this somewhat answers my question of there being a buffer on each side: the answer is it wouldn’t matter if the receiving side had a buffer; the carrier is only concerned with our local port (and we can use other network congestion mechanisms to slow down the rate we send data to the remote). Though do tell me if I’m wrong!