Slow escript when parsing binaries on OTP 26.x

Questions / Help
1

Hi guys,

Let’s compare the performance between an escript and a regular module:

escript

#!/usr/bin/env escript

main(_) ->
    {ok, Bin} = file:read_file("./large_file.txt"),
    Fun = fun() -> parse(Bin) end,
    io:format("took: ~p~n", [timer:tc(Fun)]).

parse(<<_, Rest/binary>>) ->
    parse(Rest);
parse(<<>>) ->
    ok.

First, we generate a large test file:

$ dd if=/dev/random of="large_file.txt" bs=250M count=1

This never returns:

$ ./a.escript

module

-module(a).
-export([main/0]).

main() ->
    {ok, Bin} = file:read_file("./large_file.txt"),
    Fun = fun() -> parse(Bin) end,
    io:format("took: ~p~n", [timer:tc(Fun)]).

parse(<<_, Rest/binary>>) ->
    parse(Rest);
parse(<<>>) ->
    ok.

This is fast and took less than a sec:

$ erl
Erlang/OTP 26 [erts-14.2.1] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] [jit:ns] [lock-counting] [dtrace]

Eshell V14.2.1 (press Ctrl+G to abort, type help(). for help)
1> a:main().
took: {633916,ok}
ok

I can reproduce this behavior using Erlang 26.2.1 on both Ubuntu 22.04 LTS and macOS Sonoma (XCode 15.2).

Any idea?
Thanks

2

This is because, by default escripts are interpreted, not compiled. It’s significantly slower (especially here where you call a function 250M times), but for the most of the cases it doesn’t matter. If you want escripts to be compiled, you can set -mode(compile) in the escript or pass -c option tho the escript call in shebang:
#!/usr/bin/env -S "escript -c"

4 Likes
3

@mmin you made my day. Many thanks

1 Like
4

#!/usr/bin/env -S "escript -c"

This depends on GNU extensions and won’t work in other systems. Use

#!/usr/bin/env escript

and put

-mode(compile).

somewhere near the head of your script.

3 Likes
5

Indeed true. Would it make sense to mention that in the manpage/official docs?

1 Like
6

From the escript docs:

By default, the script will be interpreted. You can force it to be compiled by including the following line somewhere in the script file:

-mode(compile).

Execution of interpreted code is slower than compiled code. If much of the execution takes place in interpreted code, it can be worthwhile to compile it, although the compilation itself takes a little while.

2 Likes
7

In Erlang/OTP 27 escripts will be compiled by default!

8 Likes
8

I meant documenting that “-S” flag is not fully portable in env docs, not in Erlang docs.

1 Like