Set max reductions per process

Questions / Help
1

Hello everyone! Recently I had an issue in my server where the CPU spiked all the way to 100%. I struggled to debug what was the root cause since there was no crash, just a lot of wasteful operations haha. I have a hunch it was either a recursive function or some pattern matching I’m doing to handle some requests.

Function I suspect:

read_string(<<0, Rest/binary>>) -> {<<>>, Rest};
read_string(<<Char, Rest/binary>>) ->
	{String, After} = read_string(Rest),
	{<<Char, String/binary>>, After}.

Which I replaced with:

read_string(Data) ->
    [String, After] = binary:split(Data, <<0>>),
    {String, After}.

I’m not 100% this was the problem because its not infinite per say (if there is no null terminator in the input it just crashes), but it does seem to use a lot of resources.

Pattern matching I suspect:

handle_cast({chat_request, 16#10, _ToPlayersID, FromPlayerID, <<"!start", 0, _Payload/binary>>}, State) ->
...

handle_cast({chat_request, 16#10, _ToPlayersID, FromPlayerID, <<"!kick ", Payload/binary>>}, State) ->
...

etc

I was able to trace down a characters_to_utf8_trap/3 call which is a function I’m not using in my code and a bunch of processes using a lot of resources and reductions:

erlang:process_info(Pid, [memory, message_queue_len, heap_size, total_heap_size]).

[{memory,2277980040},
 {message_queue_len,139},
 {heap_size,284745853},
 {total_heap_size,284745853}] 

--

[{<0.38.0>,{reductions,658096}},
 {<0.206.0>,{reductions,344034}},
 {<0.10.0>,{reductions,283457}},
 {<0.61.0>,{reductions,273556}},
 {<0.10406.0>,{reductions,217295}},
 {<0.10220.0>,{reductions,195279}},
 {<0.168.0>,{reductions,184969}},
 {<0.22.0>,{reductions,174779}},
 {<0.3.0>,{reductions,168979}},
 {<0.169.0>,{reductions,152528}}]

I think I was able to fix the issue by improving read_string but I was wondering if there is a way to set a maximum number of reductions per process before crashing to prevent these kind of issues (or maybe there is alternatives for this?)

I was reading the flags from the doc and the closest I could find was +hmax Size which I guess could work.

+hmax Size

Sets the default maximum heap size of processes to the size Size words. Defaults to 0, which means that no maximum heap size is used. For more information, see process_flag(max_heap_size, MaxHeapSize).

If you have any comments or tips on how to deal with this I would really appreciate it.

Thanks!

2

Yes, this is very wasteful. While prepending a character to a list ([Char | List]) is efficient, prepending a character to a binary (<<Char, String/binary>>) will cause the binary to be copied.

For a binary, appending to it is an efficient operation. Here is a more efficient version of read_string:

read_string(Bin) ->
    read_string(Bin, <<>>).

read_string(<<0, Rest/binary>>, Acc) ->
    {Acc, Rest};
read_string(<<Char, Rest/binary>>, Acc) ->
    read_string(Rest, <<Acc/binary, Char>>).

In the Erlang/OTP documentation, there is an Efficiency Guide. Here is a direct link to the section about binaries:

3

Hey @bjorng thanks for the reply. I checked the link and the entire performance section. Really useful!