Yet another “pool mng” library.
Sherlock is an Erlang library that provides a connection pooling mechanism for any purposes. It aims to simplify the management of database connections in a concurrent environment.
This ‘pool manager’ without singletone process called ‘manager’ and based on ETS and counters. Allow your pool survive even when it overloaded.
Yet another pool library 
I’m curious, how does it compare to the already existing ones, worker_pool and poolboy are the first ones that come to mind. I’m curious to see in detail where each specialise, what are the criteria for each to be chosen and how they can grow together considering their strengths 
Disclaimer, I’m a maintainer of worker_pool 
It looks like - this library cannot be used in rebar projects 
. Is there any plans that to change it? 
Also, based on CI it works only on Erlang/OTP 22 - is there any plans to expand supported OTP versions? Could you please also push this lib into hex.pm?
Yes, I know there are many excellent libraries that I also actively use in my work/projects. And among them are poolboy and worker_pool and episcina and some proprietary ones. That’s why I say that this is “another pool library”
There is probably no such developer who would not do his own implementation of the pool
However, there were moments that somehow did not suit me. That’s why another library was created in the desire to remove what did not suit. Well, make some mistakes of your own while trying to correct other people’s
As for the rebar - there was no such need. If necessary, of course I will add.
I have this lib working on the 25th version without problems, so I don’t quite understand what kind of support we are talking about. Could you please add some details?
Regarding Hex - I will do it as soon as I have free time.
As for the rebar - there was no such need. If necessary, of course I will add.
I suppose - this is not necessary, but make sense, because a lot of projects are use rebar. The Erlang.mk are support rebar projects, but rebar do not support Erlang.mk - here is a problem.
I have this lib working on the 25th version without problems, so I don’t quite understand what kind of support we are talking about. Could you please add some details?
As it was mention before - based on CI it supported OTP 22. If this lib already supported let’s say from 22 ~26 - it make sense add those OTP versions into matrix of CI.
Regarding Hex - I will do it as soon as I have free time.
Cool, thanks!
However, there were moments that somehow did not suit me.
Could you please describe a little bit more about it? Which moments to what libs was related and what will be achieve or what problems will solved if developers will start use Sherlock let’s say instead of of worker_pool?
Could you please describe a little bit more about it? Which moments to what libs was related and what will be achieve or what problems will solved if developers will start use Sherlock let’s say instead of of
worker_pool?
In most cases and on the vast majority of projects, there is probably no big difference which pool to use. And in my opinion, you should use what you are familiar with. Moreover, with minor loads, it is difficult and I would say impossible to notice the difference. Therefore, I would definitely not start rewriting everything in a row on Sherlock if you have a well-functioning pool.
What is sherlock about? This is an attempt to solve some issues that started to arise when using rather large pools (from 2k workers). It is also an attempt to solve some problems that arose with other pools during load (~2-10k requests per second to the pool).
How is this resolved? Sherlock, this can be called a “pool on the contrary” - where the workers themselves look for work after they are freed from the previous task. Unlike some other solutions - where after being released, the worker “reports” to the pool manager that he is free and ready to work. Sherlock makes sure that the worker before “returning to the pool” looks to see if there is a task for him. And if there is one, he immediately takes it.
It should also be borne in mind that Sherlock implements only one strategy - round robin.
And can use any term as the pool name. This was also quite an important condition on one of the projects because pools were created and deleted at runtime.
I’m curious if you had a look at pooler Pooler - the most advanced Erlang worker pool library?
It is relatively compact library (~1500LOC) and we use it in production to hold epgsql connections: it is able (in the most recent versions though) to handle up to 7k requests per second without problems (I just checked our dashboard). However we only have smth like 100 workers in the pool, had no need to have more than that; but it uses a map as main storage so should be between O(1) and O(log n) from the number of workers.
It is also quite well-tested (unit tests, property-based tests, micro-benchmarks).
workers themselves look for work after they are freed from the previous task.
does it mean that the worker process should be modified to include the logic to check if there is a work for it in the pool? Or worker process is wrapped in another process that implements this checking logic? Or I misunderstood smth?
And can use any term as the pool name. This was also quite an important condition on one of the projects because pools were created and deleted at runtime.
Cool feature! I need to think if I could steal that
For rebar3 you just need to add a .app file or .app.src file so rebar3 knows there is an Application there. No need for rebar.config file since it has no third party deps.
No I’m somehow don’t found it. I’ll read about it thank you.
We are had found (GitHub - ferd/dispcount: Erlang task dispatcher based on ETS counters.) and has been inspired that idea.
This logic is wraparound. There is no any necessary to write some additional code. All maked trasparent.
urw