I’m curious if you had a look at pooler Pooler - the most advanced Erlang worker pool library?
It is relatively compact library (~1500LOC) and we use it in production to hold epgsql connections: it is able (in the most recent versions though) to handle up to 7k requests per second without problems (I just checked our dashboard). However we only have smth like 100 workers in the pool, had no need to have more than that; but it uses a map as main storage so should be between O(1) and O(log n) from the number of workers.
It is also quite well-tested (unit tests, property-based tests, micro-benchmarks).
workers themselves look for work after they are freed from the previous task.
does it mean that the worker process should be modified to include the logic to check if there is a work for it in the pool? Or worker process is wrapped in another process that implements this checking logic? Or I misunderstood smth?
And can use any term as the pool name. This was also quite an important condition on one of the projects because pools were created and deleted at runtime.
Cool feature! I need to think if I could steal that 