# EEP 76: Priority Messages **URL:** \ **Category:** Proposals: RFC\ **Tags:** erlang-otp\ **Created:** [January 7, 2025, 10:37pm UTC](https://erlangforums.com/t/eep-76-priority-messages/4361 "2025-01-07T22:37:21Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![rickard](https://erlangforums.com/user_avatar/erlangforums.com/rickard/32/184_2.png) [@rickard](https://erlangforums.com/u/rickard)\ **Post date:** [January 7, 2025, 10:37pm UTC](https://erlangforums.com/t/eep-76-priority-messages/4361/1 "2025-01-07T22:37:21Z")
**Abstract** In some scenarios it is important to propagate certain information to a process quickly without the receiving process having to search the whole message queue which can become very inefficient if the message queue is long. [This EEP](https://www.erlang.org/eeps/eep-0076) introduces the concept of priority messages to the language which aim to solve this issue. For more information see [pull request 73](https://github.com/erlang/eep/pull/73) of the EEP repository. ---
**Author:** ![zabrane](https://erlangforums.com/letter_avatar_proxy/v4/letter/z/7ba0ec/32.png) [@zabrane](https://erlangforums.com/u/zabrane)\ **Post date:** [January 8, 2025, 7:03am UTC](https://erlangforums.com/t/eep-76-priority-messages/4361/2 "2025-01-08T07:03:59Z")
+1 for this concept. ---
**Author:** ![Maria-12648430](https://erlangforums.com/user_avatar/erlangforums.com/maria-12648430/32/604_2.png) [@Maria-12648430](https://erlangforums.com/u/Maria-12648430)\ **Post date:** [January 8, 2025, 9:59am UTC](https://erlangforums.com/t/eep-76-priority-messages/4361/3 "2025-01-08T09:59:11Z")
Thanks for taking this on @rickard, I was hoping for this for a while šŸ„° ---
**Author:** ![rvirding](https://erlangforums.com/user_avatar/erlangforums.com/rvirding/32/178_2.png) [@rvirding](https://erlangforums.com/u/rvirding)\ **Post date:** [January 8, 2025, 12:07pm UTC](https://erlangforums.com/t/eep-76-priority-messages/4361/4 "2025-01-08T12:07:12Z")
I am sure this is fantastic but I reckon that we are going to see a lot of high priority messages. I know that the messages I send are important and will have high priority. ---
**Author:** ![ieQu1](https://erlangforums.com/user_avatar/erlangforums.com/iequ1/32/2633_2.png) [@ieQu1](https://erlangforums.com/u/ieQu1)\ **Post date:** [January 8, 2025, 12:22pm UTC](https://erlangforums.com/t/eep-76-priority-messages/4361/5 "2025-01-08T12:22:58Z")
The problem of long message queue does, indeed, bite quite often, but IMO current message ordering guarantees are too nice to lose. An alternative solution that I was thinking about (but not to the point of submitting an EIP), would be to extend process alias mechanism to the receive side: 1. Add a new BIF to create a new process mailbox. This BIF returns a reference that acts similar to process alias. Semantics of mailboxes created explicitly are the same. 2. Messages sent to this reference end up in a separate mailbox. 3. Add a new language construct, `from Alias receive ...` (or similar) 4. This will allow to write code like ```erlang recieve %% Process high prio control messages from the supervisor first: {'EXIT', ...} -> terminate(...) after 0 -> %% Process other stuff: from OtherMailbox receive {'$gen_call', From, ...} -> handle_call(...) end end ``` Note: both receives should be treated as _one_ by the compiler, rather than as 2 separate receives. This solution preserves message causality within each mailbox. Additionally, it can be utilized by BEAM languages with static typing, because they could use a convention that each mailbox contains messages of the same type, but this is another story. Just throwing this idea here. I understand that it requires significant changes in ERTS, compiler and standard library (e.g. `gen:call`). ---
**Author:** ![dischoen](https://erlangforums.com/user_avatar/erlangforums.com/dischoen/32/695_2.png) [@dischoen](https://erlangforums.com/u/dischoen)\ **Post date:** [January 8, 2025, 12:40pm UTC](https://erlangforums.com/t/eep-76-priority-messages/4361/6 "2025-01-08T12:40:13Z")
I just thought of something similar, but without the need for new language constructs: - create a dedicated priority mail receiver process in a 1:1 relationship with the worker - priority messages go to the priority process, which caches them - like the ā€œfrom OtherMailboxā€ construct the worker regularly polls the priority receiver for important messages. of course the pid or name of the priority receiver has to be known to the sender(s). ---
**Author:** ![dischoen](https://erlangforums.com/user_avatar/erlangforums.com/dischoen/32/695_2.png) [@dischoen](https://erlangforums.com/u/dischoen)\ **Post date:** [January 8, 2025, 12:46pm UTC](https://erlangforums.com/t/eep-76-priority-messages/4361/7 "2025-01-08T12:46:36Z")
I think the guys at the late twitter had to implement something similar for their boss. ---
**Author:** ![Maria-12648430](https://erlangforums.com/user_avatar/erlangforums.com/maria-12648430/32/604_2.png) [@Maria-12648430](https://erlangforums.com/u/Maria-12648430)\ **Post date:** [January 8, 2025, 4:56pm UTC](https://erlangforums.com/t/eep-76-priority-messages/4361/8 "2025-01-08T16:56:18Z")
Well, the EEP says that the _receiver_ has to configure itself (via `process_flag`) to actually prioritize priority-marked messages from a specific _sender_. In other words, while you may consider your messages important for the receiver, the receiver has to consider important messages coming from you as important for him also šŸ˜‰ ---
**Author:** ![Maria-12648430](https://erlangforums.com/user_avatar/erlangforums.com/maria-12648430/32/604_2.png) [@Maria-12648430](https://erlangforums.com/u/Maria-12648430)\ **Post date:** [January 8, 2025, 5:10pm UTC](https://erlangforums.com/t/eep-76-priority-messages/4361/9 "2025-01-08T17:10:21Z")
> [@ieQu1](#): > > The problem of long message queue does, indeed, bite quite often, but IMO current message ordering guarantees are too nice to lose. If you read the EEP (or rather, the PR for it, at the time of this writing) carefully, you will see that you wonā€™t lose anything that is currently guaranteed. What you can change, if you as a receiver opt _in_ for it, and that for specific senders, is the order in which messages end up in your mailbox. That is, if and only if a process A sends a _priority_ message to process B, _and_ process B has opted in to prioritize priority messages from process A, then the message will be actually prioritized in process Bā€™s mailbox. Messages from other processes (priority or not) as well as non-priority messages even from B, will be put at the end of the mailbox as usual. ---
**Author:** ![max-au](https://erlangforums.com/user_avatar/erlangforums.com/max-au/32/1627_2.png) [@max-au](https://erlangforums.com/u/max-au)\ **Post date:** [January 9, 2025, 4:49am UTC](https://erlangforums.com/t/eep-76-priority-messages/4361/10 "2025-01-09T04:49:37Z")
Itā€™s finally happening! Thanks for formalising the design we discussed to death over emails and not only, the signal ā†’ message conversion magic. As I now read through the code, the fun part is repurposing receive marker storage when the process flag is set. ---
**Author:** ![MononcQc](https://erlangforums.com/user_avatar/erlangforums.com/mononcqc/32/417_2.png) [@MononcQc](https://erlangforums.com/u/MononcQc)\ **Post date:** [January 9, 2025, 5:05am UTC](https://erlangforums.com/t/eep-76-priority-messages/4361/11 "2025-01-09T05:05:34Z")
This looks like really risky dynamics, but since itā€™s opt-in from the receiver that makes it reasonable. Iā€™m curious about the semantics of toggling priority reception on and off while messages might be in flight, actively enqueued, etc. I also think thereā€™s potential here to make appups and relups a lot more reliable and safe by having the sys messages triggering the code update turn on priority reception for the duration of the upgrade. Part of the risks of these operations were that if they took place in busy processes, the gradual increase of mailbox size across dozens or hundreds of workers could take the node on a death spiral. High priority relup messages could lower the duration of code\_change communications and drastically de-risk these operations. ---
**Author:** ![rickard](https://erlangforums.com/user_avatar/erlangforums.com/rickard/32/184_2.png) [@rickard](https://erlangforums.com/u/rickard)\ **Post date:** [January 9, 2025, 6:34pm UTC](https://erlangforums.com/t/eep-76-priority-messages/4361/12 "2025-01-09T18:34:47Z")
> [@ieQu1](#): > > ```erlang > recieve > %% Process high prio control messages from the supervisor first: > {'EXIT', ...} -> terminate(...) > after 0 -> > %% Process other stuff: > from OtherMailbox receive > {'$gen_call', From, ...} -> handle_call(...) > end > end > > ``` > > Note: both receives should be treated as _one_ by the compiler, rather than as 2 separate receives. It is more or less the alternate solution I considered in the EEP. Iā€™m not fond of such a solution since it makes the `receive` expression very complicated and you wouldnā€™t be able to use this functionality in `gen_server`s, etc, without having to modify all behaviors as well. Using the solution proposed in the EEP, the only thing you need to do is to add the following to the initialization of the worker process that should prioritize the supervisor `'EXIT'` message: ```erlang process_flag({priority_exit_message, SupervisorPid}, true), ``` and then you are done. One thing that I think is nice about the approach in the EEP is that it solves these issues without introducing major changes. ---
**Author:** ![rickard](https://erlangforums.com/user_avatar/erlangforums.com/rickard/32/184_2.png) [@rickard](https://erlangforums.com/u/rickard)\ **Post date:** [January 9, 2025, 6:36pm UTC](https://erlangforums.com/t/eep-76-priority-messages/4361/13 "2025-01-09T18:36:32Z")
As @Maria-12648430 stated ---
**Author:** ![rickard](https://erlangforums.com/user_avatar/erlangforums.com/rickard/32/184_2.png) [@rickard](https://erlangforums.com/u/rickard)\ **Post date:** [January 9, 2025, 6:41pm UTC](https://erlangforums.com/t/eep-76-priority-messages/4361/14 "2025-01-09T18:41:31Z")
Having to poll for important events is quite annoying and often costly. The polling workarounds is what the proposed solution is trying to get away from. ---
**Author:** ![rickard](https://erlangforums.com/user_avatar/erlangforums.com/rickard/32/184_2.png) [@rickard](https://erlangforums.com/u/rickard)\ **Post date:** [January 9, 2025, 7:15pm UTC](https://erlangforums.com/t/eep-76-priority-messages/4361/15 "2025-01-09T19:15:35Z")
> [@MononcQc](#): > > Iā€™m curious about the semantics of toggling priority reception on and off while messages might be in flight, actively enqueued, etc. The state of the process-flag at the time of the signal reception determine the action taken for the message. If it is `true`, overtake ordinary messages in the message queue and add it there; otherwise, add it to the end of message queue. Once it has been added to the message queue, it wont move. Regarding prioritized code change. This is not something that I have been looking into and feels to me a bit like stretching what it was intended for, but I might be wrong, it can perhaps be solved. Just an idea (which perhaps might be stupid since I havenā€™t looked into the details here) in an initial phase (new or existing) before anything regarding the upgrade has been done send a sys-message request to the involved processes to enable priority messaging. Once replies from all involved processes have been received, initiate the upgrade. Disable priority messaging in final phase (new or existing). ---
**Author:** ![max-au](https://erlangforums.com/user_avatar/erlangforums.com/max-au/32/1627_2.png) [@max-au](https://erlangforums.com/u/max-au)\ **Post date:** [January 10, 2025, 2:12am UTC](https://erlangforums.com/t/eep-76-priority-messages/4361/16 "2025-01-10T02:12:41Z")
> [@rickard](#): > > Using the solution proposed in the EEP, the only thing you need to do is to add the following On that note, I think itā€™d be useful to have a way to override the global default (via beam command line switch). There may also be some use of that for `proc_lib:start/spawn` function family, but Iā€™d rather abstain from that. ---
**Author:** ![rvirding](https://erlangforums.com/user_avatar/erlangforums.com/rvirding/32/178_2.png) [@rvirding](https://erlangforums.com/u/rvirding)\ **Post date:** [January 10, 2025, 7:26pm UTC](https://erlangforums.com/t/eep-76-priority-messages/4361/17 "2025-01-10T19:26:07Z")
I interpreted @rickard that there wasnā€™t a global default but that you explicitly set it inside a process: ```erlang process_flag({priority_exit_message, SupervisorPid}, true), ``` ---
**Author:** ![max-au](https://erlangforums.com/user_avatar/erlangforums.com/max-au/32/1627_2.png) [@max-au](https://erlangforums.com/u/max-au)\ **Post date:** [January 10, 2025, 8:22pm UTC](https://erlangforums.com/t/eep-76-priority-messages/4361/18 "2025-01-10T20:22:51Z")
Yes, Iā€™m saying, that Iā€™d like to add the global default override to that PR, for completeness. ---
**Author:** ![dszoboszlay](https://erlangforums.com/user_avatar/erlangforums.com/dszoboszlay/32/1346_2.png) [@dszoboszlay](https://erlangforums.com/u/dszoboszlay)\ **Post date:** [January 11, 2025, 12:21am UTC](https://erlangforums.com/t/eep-76-priority-messages/4361/19 "2025-01-11T00:21:27Z")
I agree with the motivation and the see the relevance of the problematic scenarios listed in the EEP. But, to be honest, Iā€™m not entirely sold to the proposed solution. - One problem I have is that the receiver needs to opt-in for priority messages per sender. The current practice is that only the sender needs to know who will be the receiver of a message, a process generally doesnā€™t care about who will send a message to it. Perhaps in these specific problematic scenarios the receiver would know who may send a priority message to it in the future, but there are many scenarios when the sender is not known. For example it may be useful to schedule a priority message with `erlang:send_after/4`, but you donā€™t know what process to expect that message from. - The EEP deals with regular messages from a process, exit signals and monitor signals. They all need different `process_flag/2`-s for opting in, which is already a lot of complexity. `monitor/3` gets a new option to make this easier to use, but for a complete priority-message support youā€™d need similar extensions to a lot of other API-s that set up some message to be sent in the future: timers, `erlang:send_after/4`, `erlang:monitor_node/3`, `net_kernel:monitor_nodes/2`, trace messages, OS signal handling, active messages sent by `gen_tcp`, the `{tcp_passive, Socket}` message sent upon exhausting the receive window of an `{active, N}` socket and so on. A complete support of priority message (where any message you may expect from Erlang/OTP can be configured to be sent as priority) would be a huge change, with a lot of updates both in the client API-s and in the implementation of the services (so they know whether to send regular or priority messages). - A minor note is that I donā€™t see the EEP mentioning NIF-s. NIF-s would also need an API for sending priority messages. If I may propose an alternative solution to consider, I think **indexed message queues** could solve this problem with less complexity (at least in the language and library level). The idea is that a process could specify one or more patterns (in the form of a match specification) used for indexing its message queue. This could be a `process_flag/2`, for example like this: ```erlang process_flag( message_queue_indexing, [{ {'EXIT', '_', '_'}, [], [] }, % index EXIT messages { {immediate_abort, [], [] } % index some arbitrary message relevant for this process ]) ``` This call would create 2 indices for the process. When a message is placed in the message queue of a process with `message_queue_indexing`, the message is tested against the match spec, and gets inserted into all of the indices too (an index wouldnā€™t be a separate message queue, just something like a linked list of pointers to the actual messages in the one and only message queue). Now, the trick is that when the process later does a selective receive, the receive patterns could be compared with the match spec and if all of the receive patterns are indexing patterns too, then the receive would only have to scan those indices, not the entire message queue. This would help speeding up code written in the following (I believe quite common) style, without messing with message ordering guarantees in any way: ```erlang receive immediate_abort -> exit(shutdown); {'EXIT', Pid, Reason} when Pid =:= MyWorker -> restart_worker() after 0 -> receive Msg -> handle_msg(Msg) end end ``` The downside is that itā€™s obviously a harder problem to implement these indexed receives in the compiler and ERTS (e.g. the compiler would have to turn the patterns to match specifications and then ERTS would have to figure out whether the index match specification fully covers the receive match specification; not to mention maintaining the indices as messages are inserted and removed from the message queue). ---
**Author:** ![max-au](https://erlangforums.com/user_avatar/erlangforums.com/max-au/32/1627_2.png) [@max-au](https://erlangforums.com/u/max-au)\ **Post date:** [January 11, 2025, 2:53am UTC](https://erlangforums.com/t/eep-76-priority-messages/4361/20 "2025-01-11T02:53:55Z")
The way I understand your proposal, itā€™s an implementation of multiple message queues, with somewhat convoluted ā€œselective-receive-likeā€ syntax to specify the order in which receiving process wants to fetch these messages. While generally Iā€™m in favour of multi-receive-queue solutions, I think it will be quite a revolution to implement. That said, if someone can come with a decent implementation of that, it may be a really cool feature. [Next page](https://erlangforums.com/t/eep-76-priority-messages/4361.md?page=2)