How to use “SelectInfo” with gen_sctp sockets?

Questions / Help
,
1

Hi!

Some context…

  1. gen_sctp:send/4 gets an eagain error because the socket send buffer is full;
  2. eventually after some time the network congestion is over and the socket send buffer has some empty space again;
  3. thrashing (at high call rates) the operating system kernel with gen_sctp:send/4 calls until the “socket buffer full” condition is cleared is not really optimal;

Usually in these cases the socket is monitored for “write ready” events.
I see that the generic socket can be monitored in Erlang for this kind of events by means of “SelectInfo”.
https://www.erlang.org/docs/23/man/socket#send-4
https://www.erlang.org/docs/23/man/socket#type-select_info

How to use “SelectInfo” with gen_sctp sockets though?

SCTP socket API “SCTP_SENDER_DRY_EVENT” event would offer a similar functionality. Erlang gen_sctp does not seem to support it, or?
https://www.rfc-editor.org/rfc/rfc6458#section-6.1.9

Thanks a lot!
Cristian

2 Likes
2

For SCTP-specific events like SCTP_SENDER_DRY_EVENT, it seems that the gen_sctp module does not expose a direct interface for handling such events.

Given your scenario, where you want to avoid thrashing the kernel with gen_sctp:send/4 calls, you may need to implement a form of backpressure in your application logic. For example, you could use a simple buffering mechanism where you enqueue messages to be sent and periodically check if the socket is ready for writing.

Here’s a basic example to illustrate this idea:

-module(sctp_sender).
-export([start/0, send_data/2, send_loop/1]).

start() ->
    {ok, SctpSocket} = gen_sctp:open([{active, false}]),
    spawn(fun() -> send_loop(SctpSocket) end).

send_loop(SctpSocket) ->
    receive
        {send_data, Data} ->
            case gen_sctp:send(SctpSocket, 0, 0, Data) of
                {ok, _} ->
                    send_loop(SctpSocket);
                {error, eagain} ->
                    % Buffer the data and retry later
                    send_loop(SctpSocket)
            end;
        stop ->
            ok
    end.

send_data(SctpSocket, Data) ->
    SctpSocket ! {send_data, Data}.
1 Like
3

Hi,

  1. Implementing socket “polling” in the application vs. operating system & prog. language support
    All modern Unix based operating systems offer system calls for monitoring events on file descriptors (epoll, poll, select, kqueue, aso). These system calls can be used to get asynchronous notifications for events like:

  • file descriptor (socket) is ready for read();

  • file descriptor (socket) is ready for write();

For example epoll_create(), epoll_ctl(), epoll_wait() are available in libc in Linux and can be used in C based applications. A process which waits on an kernel notification for a certain event consumes less resources than one that continuously polls a resource by making periodically a system call.

As far as I see, Erlang offers select() like notifications when using socket and the following APIs:
https://www.erlang.org/doc/man/socket#send-3
with an infinite time-out. See more details here (esp. the first “Note”):
https://www.erlang.org/doc/man/socket
I assume that this language feature is implemented using system calls like epoll, select, kqueue aso.

I do not understand why gen_tcpand gen_sctp do NOT offer the same kind of APIs with select support (asynchronous calls) for their send and receive APIs. Am I missing something?
https://www.erlang.org/doc/man/gen_sctp
https://www.erlang.org/doc/man/gen_tcp

  1. Application buffering of network messages
    It usually depends on the actual application layer and its design if it makes sense to buffer messages in application buffers. If the kernel has already buffered n messages, does it make sense for your application to buffer some more m messages? Sometimes it does, sometimes it doesn’t…
    Also, modern operating systems offer APIs for controlling the size of socket send/receive buffers.

Thanks a lot,
Cristian

1 Like
4

You are writing in Erlang, not C. Stop thinking in C or any other language that just in effect exposes raw BSD sockets to the application.

In Erlang should only care about messages. Your messages are events and your edge triggers. So set your brain to ‘co-routines’ (eg. Go channels) and not ‘threads’ (eg. ‘can I go now…what about now…?’).

For recv you use an {active,N | true} (use {active,1} to explore this), it makes the need for a select() and its schematics unnecessary (and just plain baggage) as your controlling process is in effect your ‘select’ handler (for read and errors). You do not need a select() function to tell you if there is data to read as Erlang will drop the message directly into your process mailbox when it is there; really useful when you use {packet,N}. After all, if you did haveselect() telling you there is data to read you next step is going to be recv()…right?

For send described in the instructions is how to do non-blocking sends but any process can call send as the controlling process restriction is not applied which means you just offload writes to a dedicated process which sends a message back to your ‘select’ process when its mailbox is empty.

Erlang makes ioctl(Socket, nread | nwrite | nspace) mostly unnecessary.

A typical strategy in Erlang is when faced with something blocking, just wang^Wabstract it behind a process or two or three until you can ignore it. It is the Erlang equivalent of where in other languages developers would just bury a problem under eight classes and factories (or use k8s) and then just state “its the network sysadmin’s problem now” but I digress…

1 Like