Retrieve function arguments via AST

Questions / Help
, ,
1

Hello!

I’m trying to retrieve functions arguments via AST, but the method I’m using does not always return what I need.

Example 1:

example_1_test() ->
    Expr = "fun() -> fun(<<\"foo\">>, Bar, <<\"foobar\">>) -> Bar end end.",
    {ok, Tokens, _} = erl_scan:string(Expr),
    {ok, Exprs} = erl_parse:parse_exprs(Tokens),
    {value, Fun, _} = erl_eval:exprs(Exprs, []),
    FunInfo = erlang:fun_info(Fun),
    ?debugFmt("[EXAMPLE 1]~n~p~n", [FunInfo]).

$ [EXAMPLE 1]
[{pid,<0.184.0>},
 {module,erl_eval},
 {new_index,43},
 {new_uniq,<<6,83,97,170,70,21,144,209,189,241,235,98,209,166,235,8>>},
 {index,43},
 {uniq,3316493},
 {name,'-expr/6-fun-2-'},
 {arity,0},
 {env,
     [{1,[],none,none,
       #{[{clause,1,
              [{bin,1,[{bin_element,1,{string,1,"foo"},default,default}]},
               {var,1,'Bar'},
               {bin,1,[{bin_element,1,{string,1,"foobar"},default,default}]}],
              [],
              [{var,1,'Bar'}]}] =>
             {[],#{}}},
       [{clause,1,[],[],
            [{'fun',1,
                 {clauses,
                     [{clause,1,
                          [{bin,1,
                               [{bin_element,1,
                                    {string,1,"foo"},
                                    default,default}]},
                           {var,1,'Bar'},
                           {bin,1,
                               [{bin_element,1,
                                    {string,1,"foobar"},
                                    default,default}]}],
                          [],
                          [{var,1,'Bar'}]}]}}]}]}]},
 {type,local}]

The env gives me the function clauses. This is enough for me to get the function args/variables.

Example 2:

example_2_test() ->
    Expr = "mymodule:foo().",
    {ok, Tokens, _} = erl_scan:string(Expr),
    {ok, Exprs} = erl_parse:parse_exprs(Tokens),
    {value, Fun, _} = erl_eval:exprs(Exprs, []),
    FunInfo = erlang:fun_info(Fun),
    ?debugFmt("[EXAMPLE 2]~n~p~n", [FunInfo]).

foo() -> fun(<<"foo">>, Bar, <<"foobar">>) -> Bar end.

$ [EXAMPLE 2]
[{pid,<0.184.0>},
 {module,mymodule},
 {new_index,1},
 {new_uniq,<<72,79,143,91,217,115,178,85,239,21,168,227,86,130,132,25>>},
 {index,1},
 {uniq,37911674},
 {name,'-foo/0-fun-0-'},
 {arity,3},
 {env,[]},
 {type,local}]

This didn’t give me any information about clauses of the nested function.

Is there a way to get this information?
I need this information for the eel lib.
Here is my implementation for the first example, but this is not always true because of the second one.

Thanks in advance!

2 Likes
2

You could do that directly in the AST by traversing it and than calling erl_syntax:function_clauses/1 or erl_syntax:fun_expr_clauses/1 on the functions/fun_exprs you want. This would probably need some convention-based approach if you do not know exactly which function you’re gonna return (but it should work).

Another way could be to convert the resulting function into AST and than analyze it. I tried it, but erl_parser:abstract/1 returns some strange error and TBH I don’t know why.

2 Likes
3

Thanks, @mmin.
I think erl_parse:abstract/1 is what I need, but it does not parse local functions:

  Failure/Error: {error,
                        {case_clause,{type,local}},
                        [{erl_parse,abstract,3,
                             [{file,"erl_parse.yrl"},{line,1506}]},

Here is where the code throws:

abstract(Fun, A, E) when is_function(Fun) ->
    case erlang:fun_info(Fun, type) of
        {type, external} ->
            Info = erlang:fun_info(Fun),
            {module, M} = lists:keyfind(module, 1, Info),
            {name, F} = lists:keyfind(name, 1, Info),
            {arity, Arity} = lists:keyfind(arity, 1, Info),
            {'fun', A, {function,
                        abstract(M, A, E),
                        abstract(F, A, E),
                        abstract(Arity, A, E)}}
    end.

BTW, I changed the code to accept {type, local}, but the return does not help:

{'fun',0,{function,{atom,0,eel_render},{atom,0,foo},{integer,0,0}}}

I changed my code for the moment to accept only functions with arity 0 and 1. When arity 1, I pass all the view arguments as a map.
An ugly workaround to pass arguments, but this is for now until I find a solution.
Thanks again!

1 Like
4

Some extra information.
The eel transforms an HTML template to AST.
I can define something like this:

<p><%= mymodule:foo() .%></p>

I can write Erlang code between <%= and .%>.
What I need is some extra information about the mymodule:foo(). I could not resolve it if it returns a function with arguments.
For example:

mymodule:foo() ->
    fun(Foo) -> Foo end.

I know nothing about the fun(Foo) -> Foo end, but I can resolve it if there is a way to get the ‘Foo’ argument.
Also, getting the function as a string can help, but this needs to occur when the function gets called at the runtime.
Any tip, idea or some another approach is welcome.

2 Likes
5

Yeah, I see the problem. But in which case would the user want to return the function to the HTML template? I can only imagine a scenario where the user will use that function later. In that case you don’t need to know anything about the function itself, you just need to call it.

2 Likes
6

I believe functions with zero arity make no sense, but others with arguments can encapsulate some arguments to simplify the template. For example, if I want some value from a map, I can create a function as a helper to get this value without the need for ever write maps:get(foo, MyMap), instead I can have mymodule:get_foo() as get_foo() -> fun(MyMap) -> maps:get(foo, MyMap) end.

2 Likes
7

Makes sense, but I still don’t fully understand why exactly do you need argument info? Are you implementing some kind of compile-time check or?

2 Likes
8

That’s the first approach that comes to my mind, and it works if the helper function of my previous example is passed directly in the template. For example:

<p><%= fun() -> fun(MyMap) -> maps:get(foo, MyMap) end end .%></p>

This can be resolved recursively, first fun() and then fun(MyMap), but for this I need to known the MyMap.
Once I have some extra time, I will do some code revisions and think about other possibilities to solve this.

EDIT

As I mentioned, functions with arity 1 receive all bindings of the template as a map, then the helper function mentioned above can be:

get_foo() -> fun(#{'MyMap' := MyMap} = _AllBindings) -> maps:get(foo, MyMap) end.

This works in the current implementation of the eel and maybe it’s enough for now.

2 Likes
9

Aha… Your example assumes that MyMap variable is bound in the template scope. This tightly-couples variable names in template scope and in your module scope. I think this is really confusing and it’s shrinking the user’s possibilities. E.g. if the user calls my_module:get_foo() and the variable in his template is called Map then this won’t work.

IMHO assuming that anything is defined in template, besides maybe few basic things, can lead to confusion. It even complicates writing the helper functions in modules. Isn’t it more clear to write get_foo(M) -> fun(M) -> maps:get(foo,M) end.? This would also do the job if the user doesn’t have a variable called MyMap.

TLDR, variable names in the template could depend on the erlang code, but not vice versa. Passing them through variables seems more clear and safe than assuming that they are bound in the template.

4 Likes
10

I agree with you. I assume this really can be confusing. Anyway, your example get_foo(M) -> fun(M) -> maps:get(foo,M) end. will also works.
I think this is good enough for now, and I will open an issue on the eel repo referring to this discussion to take back what we said here in the future.
I will keep the code as is, and if the lib grows, this is a point to be defined.
I’m thankful for your time and the discussion. Thanks, @mmin!

2 Likes
11

There is now an issue in the eel lib about this discussion.

2 Likes
12

In template_compiler [1] we have for all called functions (filters, screen components etc.) an implicit argument Context which holds the current request context. That is the http request, language selection, time zone, request-state variables, etc.

The screen components have an extra implicit argument with the current template variables (a map with binary keys).

This keeps everything in neat functions which can be tested in isolation (and easily reused across templates).

The AST functions that we generate use special lookup functions to find the correct variables in the current variables map. Maybe you could also replace the variables in the generated AST with such lookups. That also enables sensible type casting (e.g. to lists for lists functions) and automatic resolve of special variables like the current language.

[1] GitHub - zotonic/template_compiler: Dynamic template compiler for Erlang

3 Likes
13

Hi, @mworrell! I really appreciate your comment. Your codes from Zotonic are great inspirations for me. I have run into template_compiler code while developing in my current job, but I think it does not have the response for my problem.
For example, I can have <p><%= mymodule:foo() .%></p> in the template and the erlang function in mymodule be foo() -> fun(Foo) -> Foo end.. My question is, how to know about fun(Foo) -> Foo end? I think that the only way is to read the module file and find the function and then parse it into a string. After that scan the string using erl_scan:string("mymodule:foo() -> fun(Foo) -> Foo end.") and so I can use the tokens to retrieve all information needed.

1 Like
14

If you want to know the declaration of the function, then you could also just parse the Erlang module and find all declarations. In there the function parameters should be able to be found.

We do something similar for finding translatable strings in Erlang modules.

2 Likes