EEl - Embedded Erlang - template renderer (WIP)

Hi all!

I’m working on a library like the EEx for Elixir.
It compiles and evaluates a template, e.g:

1> eel:eval(<<"Hello, <%= Name .%>!">>, #{'Name' => <<"World">>}).
<<"Hello, World!">>

Or more complex like this:
eel

It compiles the template to AST by erl_parse:parse_exprs and eval using erl_eval:exprs.

The lib repo:

I’m also working on a library on top of it. On any news, I will post about it here.

Notes

  • This library is a WIP, and ideas and help are very welcome :smiley:
  • The syntax highlighting of the screenshot comes from a fork of the vscode_erlang
19 Likes

On top of EEl, I’m creating WErl (I don’t have planned a good name for it, but this refers to “Web Erlang”). The idea is to use WebSocket to send changes to the page and only render what changed. EEl was designed thinking on this. Under the hood it uses morphdom in the front end, the same library used by Phoenix Framework, and on the server it uses cowboy as a dependency.
When the socket connects, it receives the static content as a list, then in any update sent by the server, it can resolve the dynamic content changes.

Some front code:

Partial server code:

WErl in action:
werl

This lib it’s helping me to test the EEl and maybe can have some use cases in the future.
At the moment I’m working on a code abstraction.

6 Likes

Love this. You ended up using the name I was going to use when I mused about doing this, I suppose it’s a logical conclusion.

I think if this matured enough maybe it would be a great fit for OTP, at least I can think of lot of uses for it :slight_smile:

3 Likes

I just published an extension for VSCode that provides the syntax highlighting.

2 Likes

This is really great!

I’ve been concerned about how dependent Erlang community is on erlydtl, and the development of it has somewhat stagnated. Also, Erlang syntax in the template as opposed to Django templating language is much preferred.

Pardon my misunderstanding, but will it be possible to take the AST generated from the template and put it in an erlang module like the way erlydtl does it? Combined with the rebar3 erlydtl plugin this makes it easy to deploy releases.

Looking forward to the progress on this.

1 Like

Yes, this is in my roadmap for the eel. I think something like this is really helpful.

2 Likes

There is a new feature. Now you can compile the template to a module.

eel_to_module

5 Likes

New feature available. There is a new module called eel_template. This module’s a gen_server and holds the last render information, by this, it has the capability to update only what changed.
For example:

1> {ok, Tpl} = eel:start_priv_file_template(eel, <<"greetings.html.eel">>).
{ok,<0.162.0>}

2> eel_template:render(Tpl, #{'Who' => 'Erlang', 'Msg' => <<"Hello, EEl!">>}).
{<<"<p>Me: \"Hello, Erlang!\"</p>\n<p>You: \"Hello, EEl!\"</p>">>,
#{1 => <<"Erlang">>,2 => <<"Hello, EEl!">>}}

3> eel_template:render(Tpl).
{<<"<p>Me: \"Hello, Erlang!\"</p>\n<p>You: \"Hello, EEl!\"</p>">>,
#{}}

4> eel_template:render(Tpl, #{'Msg' => <<"Hello again!">>}).
{<<"<p>Me: \"Hello, Erlang!\"</p>\n<p>You: \"Hello again!\"</p>">>,
#{2 => <<"Hello again!">>}}

The v(2) evaluates ‘Who’ and ‘Msg’ variables, v(3) none, and v(4) only the ‘Msg’.

1 Like

Ok, I’m here again after a while to talk about this lib called EEl.
Just merged a new code, I rewrote everything from scratch.
The idea is the same, but now it’s more simple and more versatile.
There is now the possibility to define a custom engine using the eel_engine behavior. The default one is the eel_smart_engine. The job of the engine is to convert the code into AST (Abstract Syntax Tree).

Compare the eel_smart_engine implementation to a simple engine used for tests.

See the README for more details, but in a nutshell, strings and files can be compiled and evaluated to binary. For example, using a simple binary as the template:

1> eel:eval(<<"Hello, <%= Name .%>!">>, #{'Name' => <<"World">>}).
[<<"Hello, ">>,<<"World">>,<<"!">>]

or compiling it to a module

1> eel:compile_to_module(<<"Hello, <%= Name .%>!">>, foo).
{ok,foo}
2> foo:eval(#{'Name' => <<"World">>}).
[<<"Hello, ">>,<<"World">>,<<"!">>]

or using the eel module functions.

Taking the module example, the is a render function that gives this result:

3> foo:render(#{'Name' => <<"World">>}).
{[<<"Hello, ">>,<<"World">>,<<"!">>],
 #{ast =>
       [{2,
         {{1,8},
          [{call,1,
               {remote,1,{atom,1,eel_converter},{atom,1,to_binary}},
               [{'fun',1,
                    {clauses,[{clause,1,[],[],[{var,1,'Name'}]}]}}]}]}}],
   bindings => #{'Name' => <<"World">>},
   changes => [{2,<<"World">>}],
   dynamic => [{2,{{1,8},<<"World">>}}],
   static => [{1,{{1,1},<<"Hello, ">>}},{3,{{1,20},<<"!">>}}],
   vars => [{2,['Name']}]}}

It returns a tuple with the binary and a metadata that I called snapshot. See for what the snapshot is useful by looking at the example in the docs, but basically, it holds information to prevent the need to recompile and gives the information to only evaluate what changed.

Note about the key of the bindings/variables passed to render/eval. Is totally fine to write keys in snake_case like #{name => <<"World">>} instead of #{'Name' => <<"World">>}, or better, #{ThisIsFine => ok} can be write as #{this_is_fine => ok}.

From time to time I see people asking about Erlang for Web Development or what framework to use. I believe this lib can help in some manner.

Just a side note writing the template directly in an Erlang module, e.g.

foo() ->
    eel:compile(<<\"
        You need to \"escape\" all \"quoted texts\" inside this binary.
        Also, you will have some indentation issues.
    \">>).

This recent proposal can help to write better multi-line texts.

Disclaimer: It is still a WIP. There are few docs, specs and tests.

And that’s it.
Suggestions welcome.

All the best,
William

6 Likes

Btw, if you possible, you should return IO list.

 eel:eval(<<"Hello, <%= Name .%>!">>, #{'Name' => <<"World">>}).

Should compile to something like:

[<<"Hello, ">>, Name, <<"!">>]

And return an IO list. This would be the most efficient approach as each static part of the template is allocated as a literal. If someone wants an actual binary, erlang:list_to_binary/1 can do the job. (I think I live streamed about this a long time ago).

This is specially important for something like WErl, because if you give each template a fingerprint (MD5 of its contents), you know you don’t need to send the static bits again.

5 Likes

Thanks for the suggestion, @josevalim.
Just merged a PR with this change and updated my comment above with the new output:

Appreciate your comments and tips!

5 Likes

I made some changes thinking in performance and, to my surprise, EEl was faster than EEx.

The evaluated templates in the benchmark was really simple:

list = Enum.to_list 1..10

Benchee.run(%{
  "eex" => fn ->
    EEx.eval_string "Foo<%= Enum.map(list, fn(x) -> %><p><%= x %></p><% end) %>Bar", [list: list]
  end,
  "eel" => fn ->
    :eel.eval("Foo<%= lists:map(fun(X) -> %><p><%= X .%></p><% end, List) .%>Bar", %{List: list})
  end
})

Of course, they are not a good parameter, but make me happy and I’ll continue dedicating my time to this lib. There is more to improve, but, in general, it’s a good start :smiley:
I’ll start to do some real use with EEl to find and fix bugs.
Give it a try if you have some interest.
Suggestions welcome! o/

5 Likes

The difference is even bigger using compiled versions:

Name           ips        average  deviation         median         99th %
eel        19.67 K       50.85 μs    ±19.27%       49.82 μs       88.78 μs
eex        15.62 K       64.04 μs    ±17.40%       63.79 μs      109.07 μs

Comparison: 
eel        19.67 K
eex        15.62 K - 1.26x slower +13.19 μs

The script:

list = Enum.to_list 1..10
eex_quoted = EEx.compile_string("Foo<%= Enum.map(list, fn(x) -> %><p><%= x %></p><% end) %>Bar")
{:ok, eel_snapshot} = :eel.compile("Foo<%= lists:map(fun(X) -> %><p><%= X .%></p><% end, List) .%>Bar")

Benchee.run(%{
  "eex" => fn ->
    Code.eval_quoted(eex_quoted, [list: list])
  end,
  "eel" => fn ->
    {:ok, render} = :eel_renderer.render(%{List: list}, eel_snapshot)
    :eel_evaluator.eval(render)
  end
})
1 Like

Can you please try running benchmarks with Mix.install([]) at the top, so protocols are consolidated? I am guessing the additional cost is pretty much us trying to convert arguments to strings.

1 Like

Sure! But this error occurs

mix run benchmark.exs
** (Mix) Mix.install/2 cannot be used inside a Mix project

If I understand correctly, I guess the same. Because of this a snapshot (metadata) is returned and not just a whole AST list in the compile function. For example, this is what the eel:compile can outputs (it’s an “older” result, I changed the snapshot to a record(), I’ll submit this change in my next PR)

{["<html><head><title>",<<"Hey!">>,
  "</title></head><body>Hello, ",<<"World">>,
  "!</body></html>"],
 #{ast =>
       [{2,
         {{1,20},
          [{call,1,
               {remote,1,{atom,1,eel_converter},{atom,1,to_string}},
               [{'fun',1,
                    {clauses,[{clause,1,[],[],[{var,1,'Title'}]}]}}]}]}},
        {4,
         {{1,61},
          [{call,1,
               {remote,1,{atom,1,eel_converter},{atom,1,to_string}},
               [{'fun',1,
                    {clauses,[{clause,1,[],[],[{var,1,...}]}]}}]}]}}],
   bindings => #{'Name' => <<"World">>,'Title' => <<"Hey!">>},
   changes => [{2,<<"Hey!">>},{4,<<"World">>}],
   dynamic =>
       [{2,{{1,20},<<"Hey!">>}},{4,{{1,61},<<"World">>}}],
   static =>
       [{1,{{1,1},"<html><head><title>"}},
        {3,{{1,33},"</title></head><body>Hello, "}},
        {5,{{1,73},"!</body></html>"}}],
   vars => [{2,['Title']},{4,['Name']}]}}

The static values (strings) are nevermore evaluated, just zipped with the new dynamic result (AST).
I guess EEl eval can be even faster because the snapshot holds the dynamic values as a cache and only the changes are evaluated.
The render function returns {iodata(), snapshot()}, iodata() is the string and snapshot() is the updated metadata that helps the next render to be faster.

2 Likes

Ah, perfect, so protocols are already consolidated. Thanks!

2 Likes