function_clause in lists:foldl during live upgrade

Questions / Help
1

Hi ,

I need help. I can’t figure it out.
We do live upgrades, and during the testing a weird error occurred.

{function_clause, [{lists,foldl, [#Fun<wep_workflow_executor.8.100719303>, #{{{mddb_wfm_check_endpoint,1},mddb_wfm_check_endpoint} => #{}, {{mddb_wfm_convert_input,2},mddb_wfm_convert_input} => #{}, {{mddb_wfm_gather_input,4},mddb_wfm_gather_input} => #{}, {{mddb_wfm_get,5},mddb_wfm_get} => #{}, {{mddb_wfm_redirect,3},mddb_wfm_redirect} => #{}}, []], [{file,"lists.erl"},{line,1266}]},

If I understand it correctly at some point is_function(F, 2) should return false to get this error.

foldl(F, Accu, [Hd|Tail]) -> foldl(F, F(Hd, Accu), Tail); foldl(F, Accu, []) when is_function(F, 2) -> Accu.

The thing is that is_function/2 gave me true even if the function not yet exists, hard purged from the vm.

I cannot reproduce this error, but already happened twice during live upgrade.
Any guess or idea what i am doing wrong?

1 Like
2

may be you need show your code. one of the reason may be you sotre the Fun but when code upgraded the fun code has been clean

2 Likes
3

It looks like your foldl is using a local function from the wep_workflow_executor module during the live upgrade. That’s a recipe for problems. Use export funs if you need them to live across live upgrades. (You may need to declosurize the fun, but since it’s passed to a foldl that’s easy.)

2 Likes
4

The thing that bothers me that i’m getting function_clause instead of bad fun.

In sequential code where no full qualified call i thought i should not be worried about code change. I mean it is not hard purged until there is a process which using that version of the module.

Security team do not let us share code, but looks something like this:

fun_with_the_problem(Param1, Param2) ->
  Map = local_fun(Param1),
  lists:foldl(fun(Ref, Acc) ->
                maps:merge(Acc, #{Ref => other_module:magic_stuff(Param2)})
              end, #{}, Map).
1 Like
5

I using these code to test:

make_fun() ->
    fun() -> io:format("aaa~n", []) end.% modify aaa to bbb in 10000

load once will use the old version code, everything is ok
load twice get an error, just like what you say before

172> spawn(fun() -> F = test:make_fun(), timer:sleep(10000), io:format("===~p~n", [erlang:is_function(F,0)]), F() end).
<0.608.0>
173> c(test).
{ok,test}
174> c(test).
{ok,test}
175> ===true
175>
=ERROR REPORT==== 29-Jun-2022::17:15:30 ===
Error in process <0.608.0> with exit value:
{{badfun,#Fun<test.1.1737472>},
 [{shell,apply_fun,3,[{file,"shell.erl"},{line,901}]}]}

so, If it’s possible:

  1. F point address X
  2. reload code twice, but process will not be kill(not calling function in test)
  3. memory not enought, reuse address X
  4. F change to something X point to
  5. is_function/2 return false

but I don’t know how to test it, just a conjecture
hope it can help you :slightly_smiling_face:

2 Likes
6

Exactly why is it a recipe for problems? I played around with code like this:

-module(up).

-export([ll/0,
            el/0,
            ex2/1]).

ll() ->
    lists:map(
      fun(N) ->
              io:format("~p ", [N]),
              timer:sleep(2000)
      end, lists:seq(1,10)),
    io:format("~n").

el() ->
    lists:map(fun ?MODULE:ex2/1, lists:seq(1,10)),
    io:format("~n").

ex2(N) ->
    io:format("~p ", [N]),
    timer:sleep(2000).

The ll “test case” uses a local function, the el “test case” uses an exported function. I start the tests like this:

spawn(fun up:ll/0)

If I change the internal io:format to io:format("~p ", [2*N]) and reload the code while the spawned process is still running, the code will still print “1 2 3 4 5 6 7 8 10”. If I do the same with the el, the code will start to execute the updated function as soon as the module is reloaded and the code will print something like this: “1 2 3 4 10 12 14 16 18 20”. It depends on the use case if the first or the second output is expected and acceptable. In both cases the spawned process dies if the module is reloaded again. Is there something else that I’m missing?

1 Like