Hello everybody, I just want to know if the Kernel Polling functions (
epoll for Linux and
kqueue for FreeBSD) are used by default to handle sockets and connections when the Kernel Poll Space is enabled in Erlang/OTP ?
I asked this questions because I had read the code source of TCP Section of Erlang Network Driver (
inet_drv.c ) and there was any Kernel Poll Function Call so I thinked that the OS use the Kernel Polling by default when creating , binding, listening and accepting sockets if the Kernel Poll Space is enabled.
inet_drv.c is a Port Driver which interacts with the BEAM’s internal I/O system by calling
driver_select. Normally, there is no direct interaction with the underlying OS polling system within a Port Driver.
However, if you follow the call path for
driver_select, you’ll find that
erl_check_io.c contains most of the I/O handling for the BEAM and that
erts/emulator/sys/win32/erl_poll.c contain the OS-specific parts.
There is a good comment at the top of
erts/emulator/sys/common/erl_poll.c which describes the fallback behavior:
* The interface is currently implemented using:
* - select
* - poll
* - /dev/poll
* - epoll with poll or select as fallback
* - kqueue with poll or select as fallback
In older versions, there used to be an
erl +K true flag to enable or disable kernel polling at runtime, but today it looks like it’s something determined more at compile-time.
As of OTP 24, you can run
erlang:system_info(check_io) to see information about the BEAM’s internal I/O checkers.
thank you very much,
FIRST : the driver
tcp_inet is reponsible for handling all Erlang tcp requests, this driver’s Entry is defined in
inet_drv.c so we can say this way that this C program is directly used by ERTS
driver_select is used in
inet_drv.c only when Defining
win32 and not in POSIX systems (Unix and Linux)
Iam sure you have great understood to internal ERTS but here you didn’t specify in more details how that works
At this instant, I had read the
inet_drv.c code from creating a listen socket to accept a connection on it and create a new connected socket and there is no
sock_select call, so as my knowledge (Iam not very experimented with C language) the New Accepted Socket’s FD should added to the Kernel Pollset immediately when accepting the connection ?
I had not read the code yet for sending and receiving data via the socket
gen_tcp:accept/1,2, see how
sock_select is only called if accepting on the socket results in
For receiving there are a couple options: (1) use
gen_tcp:recv/2,3 or (2) use
active set. The Port Driver’s
tcp_inet_drv_input callback will be called as it is defined as the callback for
ready_input in the TCP driver entry. See the case section for
sock_select is called only if
socket==INVALID_SOCKET when accepting a socket fails and returns (-1) but not called when
accept() succeed and returns FD so what you can say about that?
I have just read the code until accepting a socket like I said, so send and receiving data not yet
That’s just the way non-blocking
accept(2) functions in Linux, FreeBSD, Windows, etc.
If there’s a connection ready to accept, then it will return a new file descriptor.
If there are no connections available, it returns
errno gets set to
EWOULDBLOCK; this is the case where epoll (on Linux) would then be used as a result of
sock_select so the Port Driver can be notified once there is a connection ready to accept.
Yeah I said that you are great thank you, I think that I understand know, I did a look to the
gen_tcp:recv function track and I think that it works the same way as
accept so first the Thread check if there is any ready data to read from the socket if it’s true he will return the data using
recv() system call but if there is no data he will call
sock_select that result in adding the socket’s FD to the Kernel Pollset and the ERTS will be notified by the Kernel (using
kqueue) if there is any data arrives to this socket.
1-Iam confused about
accept, in the case of receiving data that’s ok like I explained above but in the case of accepting connections, if there is no connections how can using the Kernel Polling on just one FD (the listen socket) so as my knowledge
kqueue used on a set of FD.
2-Last thing, how can you know that
driver_select results in using
driver_select is used from the header
erl_driver.h and as my knowledge the library implementing this header is not open source code ?
The implementation of
driver_select is in
erts/emulator/sys/common/erl_check_io.c and it’s open-source along with the rest of OTP.
epoll specifics, check out the functions called by
erts/emulator/sys/common/erl_poll.c. Much of the
epoll specific parts are dependent on the C macro definitions that are determined during the
./configure step when building OTP on a Linux host. So, as long as the build system was able to detect
epoll support, it should be used, but if not then poll or select will be used as a fallback.
In my opinion:
inet_drv.c might be more difficult to understand if you’re trying to connect it to the underlying non-blocking I/O functionality compared with the
socket module with
nowait. It might be helpful to do some experiments with listening and accepting a TCP connection using
socket to see how it mirrors the underlying non-blocking OS functionality.
Thanks again, I know the
socket module and it’s lowest level than
gen_XXP but it’s recommended in the above of the module to use
gen_tcp instead of and all Erlang Softwares that I know use
gen_XXp so tracking it will be more helpful, but my problem is not at the Erlang part but in C implepentation, I think that I understand a lot of internal working of the ERTS and as I said my last thing is using Kernel Polling (
kqueue) to do non-blocking accepts in One Listen Socket (One FD), as my knowledge Kernel Polling use a Set of FDs that can updated and check periodically any news about each FD of the set if there’s news the Kernel behaves, but what exactly happens in non-blocking accepts when using Kernel Polling ? this is very important for me and if you can answer this I will be very thankful Sir.