Hi everybody, I noted that the earlier versions of Erlang/OTP use just Scheduler Threads and the total number of Threads equal to the number of available Cores, but the last versions use many other Threads I just want to know WHY ? or in other terms what are the real benefits because I can’t see any one.
In fact I mean if we have “N” cores in the hardware that means that we can run “N” Threads in PARALLEL and if we use “N+M” Threads that means that “M” Threads are always waiting for Schedule and run in SEQUENTIAL with the “N” Threads, so what are the benefits ?
Early version of the runtime system with SMP support would have to suspend all schedulers but one when doing certain operations, such as loading and unloading code, or turning off and on tracing. That was necessary because certain system resources don’t have any locks protecting them (taking locks would kill system performance, for example if it would be necessary to take a lock for each call to another module).
Later versions of the runtime systems does code loading without having to suspend any scheduler threads. The new way of doing that added a new requirement: a scheduler thread must not spend a long time executing any single Erlang process. If a scheduler thread would spend a long time in a NIF, for example, it would not be possible for another process to load code (for example) during that time.
The other threads are there to do jobs of indefinite duration or known to take a long time, for example a garbage collection of an Erlang process with a huge heap.
Thank you I think this is a convincing answer, this is about Dirty Schedulers to take tasks that last more than 1 seconde, but how about IO Polling ? Polling will not suspend the other Schedulers and can be done by the Scheduler in question perfectly, so why using another Thread ?
And why an active socket will use the main Scheduler thread to poll instead of the IO poll thread ? Just I don’t understand how can these Erlang/OTP engines help in performance when comparing to the earlier versions (I don’t talk about Dirty tasks)
Technical details: Our original approach was to simply spread the I/O
polling to each scheduler so that all schedulers were responsible for
polling of fd’s of ports and nifs that executed on them. This had the
side-effect of making it much slower to wake up schedulers to do work
(because now all scheduler sleep in poll rather than on a futex). Because
of this we have decided to move the I/O polling from the scheduler threads
to separate thread(s).
Add a new pollset that is made to handle sockets that use {active, true} or {active, N}. The new pollset will not be polled by a pollthread, but instead polled by a normal scheduler.
This change was made because of the overhead associated with constantly having to re-apply the ONESHOT mechanism on fds that all input events were interesting.