# The Many-to-One Parallel Signal Sending Optimization (blog post) **URL:** \ **Category:** Erlang Blog\ **Tags:** optimization\ **Created:** [November 5, 2021, 5:56pm UTC](https://erlangforums.com/t/the-many-to-one-parallel-signal-sending-optimization-blog-post/475 "2021-11-05T17:56:43Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![kjellwinblad](https://erlangforums.com/user_avatar/erlangforums.com/kjellwinblad/32/176_2.png) [@kjellwinblad](https://erlangforums.com/u/kjellwinblad)\ **Post date:** [November 5, 2021, 5:56pm UTC](https://erlangforums.com/t/the-many-to-one-parallel-signal-sending-optimization-blog-post/475/1 "2021-11-05T17:56:43Z")
I have just posted [a blog post](https://www.erlang.org/blog/parallel-signal-sending-optimization/) about a new signal sending optimization (that most likely will be included in Erlang/OTP 25) to the [erlang.org](https://www.erlang.org) blog. The optimization makes sending signals from multiple processes to a single process much more scalable on multicore systems. Comments are very welcome! ---
**Author:** ![OvermindDL1](https://erlangforums.com/user_avatar/erlangforums.com/overminddl1/32/398_2.png) [@OvermindDL1](https://erlangforums.com/u/OvermindDL1)\ **Post date:** [November 5, 2021, 7:18pm UTC](https://erlangforums.com/t/the-many-to-one-parallel-signal-sending-optimization-blog-post/475/2 "2021-11-05T19:18:05Z")
I just saw that post, looks fascinating! I’m curious what caused that issue to be looked at so this fix came about? Is there a backstory? ---
**Author:** ![kjellwinblad](https://erlangforums.com/user_avatar/erlangforums.com/kjellwinblad/32/176_2.png) [@kjellwinblad](https://erlangforums.com/u/kjellwinblad)\ **Post date:** [November 6, 2021, 6:16am UTC](https://erlangforums.com/t/the-many-to-one-parallel-signal-sending-optimization-blog-post/475/3 "2021-11-06T06:16:56Z")
Excellent question. Thank you! The optimization was not triggered by an actual application where we found that the signal queue was a scalability bottleneck. I don’t know know if such applications exist, but I would not be surprised if they do. One type of application that might benefit from the optimization is logging systems, where one process aggregates log entries coming from multiple processes. It would be interesting to hear about it if someone knows any real-world application where the signal queue might be a scalability bottleneck. As I also mention in the blog post, people may have avoided writing applications in a certain way to avoid scalability problems with the signal queue. Now, with the new optimization, it is possible to write applications differently that might be more efficient. Also, even if the signal queue is not a scalability problem today, it might be in the future if processes keep getting more cores. I am interested in synchronization algorithms and have a background in doing academic research about such algorithms. Therefore, I was thinking about which places in the Erlang VM could have contended hot-spots. After some discussion with @rickard, I realized that there was a potential to make signal sending scale much better with the number of processes that send signals to a single process. This is how the optimization came to be 🙂 ---
**Author:** ![domi](https://erlangforums.com/user_avatar/erlangforums.com/domi/32/263_2.png) [@domi](https://erlangforums.com/u/domi)\ **Post date:** [November 6, 2021, 8:44am UTC](https://erlangforums.com/t/the-many-to-one-parallel-signal-sending-optimization-blog-post/475/4 "2021-11-06T08:44:25Z")
There was a presentation about hitting the limits of the standard Elixir logger last year: [Adventures in High Performance Logging (codesync.global)](https://codesync.global/media/adventures-in-high-performance-logging-cbv2020-sept/) I’d be curious how much a difference this makes! The benchmarks are impressive. Really appreciate the blog post detailing how this all works too. ---
**Author:** ![phild](https://erlangforums.com/user_avatar/erlangforums.com/phild/32/742_2.png) [@phild](https://erlangforums.com/u/phild)\ **Post date:** [November 7, 2021, 11:49am UTC](https://erlangforums.com/t/the-many-to-one-parallel-signal-sending-optimization-blog-post/475/5 "2021-11-07T11:49:00Z")
> [@kjellwinblad](#): > > real-world application where the signal queue might be a scalability bottleneck I did some preliminary design work earlier this year (sadly the project was subsequently descoped) for an Erlang MME component based around an in-memory database and stateless microservices. The interfaces would have had a many-to-one relationship with the database and this was indeed a concern. ---
**Author:** ![OvermindDL1](https://erlangforums.com/user_avatar/erlangforums.com/overminddl1/32/398_2.png) [@OvermindDL1](https://erlangforums.com/u/OvermindDL1)\ **Post date:** [November 8, 2021, 5:36pm UTC](https://erlangforums.com/t/the-many-to-one-parallel-signal-sending-optimization-blog-post/475/6 "2021-11-08T17:36:27Z")
Nice, thanks for the followup! The speedup is extremely impressive I have to say, looking forward to looking at using that in the future. ---
**Author:** ![starbelly](https://erlangforums.com/user_avatar/erlangforums.com/starbelly/32/209_2.png) [@starbelly](https://erlangforums.com/u/starbelly)\ **Post date:** [November 8, 2021, 7:46pm UTC](https://erlangforums.com/t/the-many-to-one-parallel-signal-sending-optimization-blog-post/475/7 "2021-11-08T19:46:02Z")
What do you think this could mean for message passing between nodes? The article focuses on processes within a single node, but one of my immediate thoughts was a knock on effect. Do you have any thoughts you can share on that? ---
**Author:** ![max-au](https://erlangforums.com/user_avatar/erlangforums.com/max-au/32/1627_2.png) [@max-au](https://erlangforums.com/u/max-au)\ **Post date:** [November 10, 2021, 4:32am UTC](https://erlangforums.com/t/the-many-to-one-parallel-signal-sending-optimization-blog-post/475/8 "2021-11-10T04:32:39Z")
I haven’t tried it yet, but would it also work for messages sent over Erlang distribution? This use-case (a single gen\_server acting as dispatcher and sending work items to other processes) is very standard implementation of a worker pool. ---
**Author:** ![starbelly](https://erlangforums.com/user_avatar/erlangforums.com/starbelly/32/209_2.png) [@starbelly](https://erlangforums.com/u/starbelly)\ **Post date:** [November 10, 2021, 6:15am UTC](https://erlangforums.com/t/the-many-to-one-parallel-signal-sending-optimization-blog-post/475/9 "2021-11-10T06:15:51Z")
^^ that’s the knock on effect I was getting at 😀 ---
**Author:** ![kjellwinblad](https://erlangforums.com/user_avatar/erlangforums.com/kjellwinblad/32/176_2.png) [@kjellwinblad](https://erlangforums.com/u/kjellwinblad)\ **Post date:** [November 10, 2021, 6:57am UTC](https://erlangforums.com/t/the-many-to-one-parallel-signal-sending-optimization-blog-post/475/10 "2021-11-10T06:57:12Z")
No, currently, the optimization only works for signals coming from processes on the same node. All other signals are mapped to the same slot in the buffer array. My thinking was that the optimization would be less relevant for other kinds of signals as (I think) they are rarer. I would be happy if someone who knows more about how the Erlang distribution works could write about whether the optimization could be made relevant for signals coming from other nodes or if other bottlenecks are more severe for such signals. Otherwise, I will look into this a little closer and come back with an answer when I know more. ---
**Author:** ![bjorng](https://erlangforums.com/user_avatar/erlangforums.com/bjorng/32/174_2.png) [@bjorng](https://erlangforums.com/u/bjorng)\ **Post date:** [November 10, 2021, 12:15pm UTC](https://erlangforums.com/t/the-many-to-one-parallel-signal-sending-optimization-blog-post/475/11 "2021-11-10T12:15:17Z")
Not my area of expertise, but wanting to learn more I talked with the team. The same optimization should be possible to do (and would probably not be hard to do) for messages coming from other nodes. Whether it is worth it is another thing, because the receiving process must also decode the received message from the external term format, and it could very well be that the time for doing that would dominate the costs for taking locks. ---
**Author:** ![max-au](https://erlangforums.com/user_avatar/erlangforums.com/max-au/32/1627_2.png) [@max-au](https://erlangforums.com/u/max-au)\ **Post date:** [November 10, 2021, 11:58pm UTC](https://erlangforums.com/t/the-many-to-one-parallel-signal-sending-optimization-blog-post/475/12 "2021-11-10T23:58:53Z")
That’s indeed what we observed! So we don’t send the message over dist. Instead, we use `term_to_binary` on the sender side, and dispatcher process does not do the decoding. It routes the binary to worker process, which will then does `binary_to_term`. Binaries are (usually) ref-counted, so there is no visible degradation. With OTP 23 and remote spawn I found another way to achieve the desired behaviour. I reversed the flow, and sender uses remote spawn to start a new worker process, which then in turn check out necessary resources from the pool (e.g. database connection or file handle). However this approach has a drawback - there is no built-in queuing/backpressure/load shedding mechanism on the receiver side. I have a bolt-on solution (currently pending corporate scrutiny to have it published), introducing extra processes on both sender and receiver side. ---
**Author:** ![afa](https://erlangforums.com/user_avatar/erlangforums.com/afa/32/298_2.png) [@afa](https://erlangforums.com/u/afa)\ **Post date:** [November 29, 2021, 9:40am UTC](https://erlangforums.com/t/the-many-to-one-parallel-signal-sending-optimization-blog-post/475/13 "2021-11-29T09:40:47Z")
Am I reading correctly that this will increase the throughput of `gen_servers` in case of multiple processes doing gen\_server `calls`? If yes, an application type to likely benefit seems to be message brokers, where messages from different session contexts can arrive at a single `gen_server` doing stuff like routing, storing messages, logging. Depends on the respective architecture, of course, but there are surely things like this in VerneMQ. ---
**Author:** ![kjellwinblad](https://erlangforums.com/user_avatar/erlangforums.com/kjellwinblad/32/176_2.png) [@kjellwinblad](https://erlangforums.com/u/kjellwinblad)\ **Post date:** [December 14, 2021, 7:01am UTC](https://erlangforums.com/t/the-many-to-one-parallel-signal-sending-optimization-blog-post/475/14 "2021-12-14T07:01:47Z")
The current gen\_server implementation uses a single process (under the hood) to handle requests that are coming as normal messages so I think this optimization could benefit gen\_servers that are used by several processes. However, the optimization is only activated for processes with the {message\_queue\_data, off\_heap} setting, so if the current gen\_server implementation takes advantage of the optimization depends on how its server process is configured. Perhaps someone that knows the gen\_server implementation better than I do can answer if {message\_queue\_data, off\_heap} is activated by default for gen\_servers or if it is possible to configure a gen\_server to use {message\_queue\_data, off\_heap} (@rickard @garazdawi). ---
**Author:** ![garazdawi](https://erlangforums.com/user_avatar/erlangforums.com/garazdawi/32/175_2.png) [@garazdawi](https://erlangforums.com/u/garazdawi)\ **Post date:** [December 14, 2021, 7:23am UTC](https://erlangforums.com/t/the-many-to-one-parallel-signal-sending-optimization-blog-post/475/15 "2021-12-14T07:23:55Z")
> [@kjellwinblad](#): > > Perhaps someone that knows the gen\_server implementation better than I do can answer if {message\_queue\_data, off\_heap} is activated by default for gen\_servers or if it is possible to configure a gen\_server to use {message\_queue\_data, off\_heap} (@rickard @garazdawi). It is not activated by default, but it can be passed in the [spawn\_opts Option](https://www.erlang.org/doc/man/gen_server.html#start_link-4) when starting a gen\_server or it can be set at run-time using [process\_flag](https://www.erlang.org/doc/man/erlang.html#process_flag_message_queue_data). ---
**Author:** ![pablopla](https://erlangforums.com/user_avatar/erlangforums.com/pablopla/32/1270_2.png) [@pablopla](https://erlangforums.com/u/pablopla)\ **Post date:** [February 17, 2022, 9:26pm UTC](https://erlangforums.com/t/the-many-to-one-parallel-signal-sending-optimization-blog-post/475/16 "2022-02-17T21:26:42Z")
I have 1 gen\_server receiving messages from 100 processes and broadcasting every message to 1,000 processes. Each of the 100 processes sends 100 messages per second to the gen\_server. Will using off\_heap help on the gen\_server help with receiving but slow down sending to all other processes? Should I set`message\_queue\_data=off\_heap on the gen\_server? Should I set it on the 100 sender processes or 1,000 receiver processes? ---
**Author:** ![kjellwinblad](https://erlangforums.com/user_avatar/erlangforums.com/kjellwinblad/32/176_2.png) [@kjellwinblad](https://erlangforums.com/u/kjellwinblad)\ **Post date:** [May 4, 2022, 2:04pm UTC](https://erlangforums.com/t/the-many-to-one-parallel-signal-sending-optimization-blog-post/475/17 "2022-05-04T14:04:36Z")
> [@pablopla](#): > > Will using off\_heap help on the gen\_server help with receiving but slow down sending to all other processes? `off_heap` could make receiving messages more efficient (how efficient is compared to `on_heap` depends on the number of cores the machine has and how often messages are sent in parallel etc). My guess is that `off_heap` will be beneficial for the gen\_server in the scenario you describe but the only way to know for sure is to measure. No `off_heap` should not affect the performance of sending to other processes. Note that the optimization described in the blog post is only available in Erlang/OTP 25.