What is the difference between "schedulers" and "schedulers online"?

Questions / Help
,
1

I’m running Elixir services in docker containers managed by Kubernetes. I see in Erlang 23 online schedulers will take CPU quotas into account (which is awesome).

If I grab an iex shell in one of these containers I see that “schedulers online” == 3, the number of CPU shares allocated, as I would expect. I also see the straight “schedulers” value is 96, which I presume is the number of cores the server hosting this container has.

Is it ok that “schedulers online” and “schedulers” do not match? Is the idea that “schedulers” is the maximum the BEAM could use, but schedulers online is the maximum it will use?

Basically I just want to feel a little more certain on the topic. I think I know what’s going on based on the docs I’ve ready but I’d like to feel confident I’m understanding correctly, so I’m here to check my understanding :slight_smile:

1 Like
2

If you go look here there’s a healthy amount of detail (look for Schedulers:SchedulerOnline)

In the spirit of time, here is the verbiage :

+S Schedulers:SchedulerOnline

Sets the number of scheduler threads to create and scheduler threads to set online. The maximum for both values is 1024. If the Erlang runtime system is able to determine the number of logical processors configured and logical processors available, Schedulers defaults to logical processors configured, and SchedulersOnline defaults to logical processors available; otherwise the default values are 1. If the emulator detects that it is subject to a CPU quota, the default value for SchedulersOnline will be limited accordingly.

Schedulers can be omitted if :SchedulerOnline is not and conversely. The number of schedulers online can be changed at runtime through erlang:system_flag(schedulers_online, SchedulersOnline).

If Schedulers or SchedulersOnline is specified as a negative number, the value is subtracted from the default number of logical processors configured or logical processors available, respectively.

Specifying value 0 for Schedulers or SchedulersOnline resets the number of scheduler threads or scheduler threads online, respectively, to its default value.

2 Likes
3

I actually found this! But the question I was left with was maybe more about the mechanics, “SchedulersOnline” sounds like “number of schedulers online right now”, so I found myself wondering if it can change over time.

Is there a situation where SchedulersOnline could increase for some reason (outside a literal config change)? Or is “SchedulersOnline” saying “here’s the number of schedulers, this number isn’t going to change unless you make it change”?

1 Like
4

Ahh, I see.

That is correct. The initialization of schedulers (cpu, io, both dirty and non dirty, etc.) and run queues is done at boot in the initialization of the emulator, after which it is on you to increase (and decrease) as needed.

5

Ahhh ok, that’s the answer I needed. Thanks again @starbelly :smile:

1 Like
6

To be clear, if you make a config change outside of the VM, just restart your container :slight_smile:

Also, FWIW are some links to key points in the boot sequence :

A starting point : erl_start

And we see a call to early_init

Then quite a ways down we call erl_init

This in turn calls erts_init_scheduling

Sorry, don’t have time right now to annotate, but if I do find some time later I will (no promises though) :slight_smile:

It would also be fantastic to hear from members from the OTP team on any minor details related to all this and whether a documentation improvement could be made per commentary.

1 Like
7

Oooh interesting! I appreciate the deep dive, I just walked through the files you shared.

1 Like