Erlang OTP 26.0 Released

Erlang News & Announcements Erlang News
1

Erlang/OTP 26.0 is a new major release with new features, improvements as well as a few incompatibilities.

Below are some of the highlights of the release:

There is also a Blogpost about the highlights.

Highlights

Parsetools

  • Leex has been extended with optional column number support.

Stdlib

  • The family of enumeration functions in module lists has been extended with enumerate/3 that allows a step value to be supplied.
  • Update Unicode to version 15.0.0.
  • proc_lib:start*/* has become synchronous when the started process fails. This requires that a failing process use a new function proc_lib:init_fail/2,3, or exits, to indicate failure. All OTP behaviours have been fixed to do this.

The Shell

There are a lot of new features and improvements in the Erlang shell:

  • auto-complete of variables, record names, record fields names, map keys, function parameter types and filenames.
  • Open external editor in the shell to edit the current expression.
  • defining records (with types), functions, specs and types in the shell.

New terminal

  • The TTY/terminal subsystem has been rewritten. Windows users will notice that erl.exe has the same functionality as a normal Unix shell and that werl.exe is just a symlink to erl.exe. This makes the Windows Erlang terminal experience identical to that of Unix.

Compiler and JIT optimizations:

  • Creation and matching of binaries with segments of fixed sizes have been optimized.

  • Creation and matching of UTF-8 segments have been optimized.

  • Appending to binaries has been optimized.

  • The compiler and JIT now generate better code for creation of small maps where all keys are literals known at compile time.

  • Thanks to the optimizations above the performance of the base64 module has been significantly improved. For example, on an x86_64 system with the JIT both encode and decode are almost three times faster than in Erlang/OTP 25.

Maps

  • Map comprehensions as suggested in EEP 58 has now been implemented.

  • Some map operations have been optimized by changing the internal sort order of atom keys.
    This changes the (undocumented) order of how atom keys in small maps are printed and returned by maps:to_list/1 and maps:next/1. The new order is unpredictable and may change between different invocations of the Erlang VM.

  • Introducing the new function maps:iterator/2 for creating an interator that return the map elements in a deterministic order.
    There are also new modifiers k and K for the format string in io:format() to support printing map elements ordered.

Dialyzer

  • Added the new built-in type dynamic() introduced in EEP 61, PR introducing EEP 61 improving support for gradual type checkers.

  • Dialyzer has a new incremental mode that be invoked by giving the --incremental option when running Dialyzer.
    This new incremental mode is likely to become the default in a future release.

Misc ERTS, Stdlib, Kernel, Compiler

  • Multi time warp mode is now enabled by default.
    This assumes that all code executing on the system is time warp safe.

  • Support for UTF-8 atoms and strings in the NIF
    interface including new functions enif_make_new_atom, enif_make_new_atom_len and enif_get_string_length.

  • The BIFs min/2 and max/2 are now allowed to be used in guards and match specs.

  • Improved the selective receive optimization, which can now be enabled for references returned from other
    functions. This greatly improves the performance of gen_server:send_request/3, gen_server:wait_response/2, and similar functions.

  • New trace feature call_memory. Similar to call_time tracing, but instead of measure accumulated time in traced functions it measures accumulated heap space consumed by traced functions. It can be used to compare how much different functions are contributing to garbage collection being triggered.

  • It is no longer necessary to enable a feature in the runtime system in order to load modules that are using it.
    It is sufficient to enable the feature in the compiler when compiling it.

  • inet:setopts/2 has got 3 new options: reuseport, reuseport_lb and exclusiveaddruse.

  • Fix so that -fno-omit-frame-pointer is applied to all of the Erlang VM when using the JIT so that tools, such as perf, can crawl the process stacks.

  • In the lists module, the zip family of functions now takes options to allow handling lists of different lengths.

  • Added the zip:zip_get_crc32/2 function to retrieve the CRC32 checksum from an opened ZIP archive.
    gen_server optimized by caching callback functions

  • The modules Erlang DNS resolver inet_res and helper modules have been updated for RFC6891; to handle OPT RR with DNSSEC OK (DO) bit.

  • Introduced application:get_supervisor/1.

  • Cache OTP boot code paths, to limit how many folders that are being accessed during a module lookup. Can be disabled with -cache_boot_path false.

SSL

  • Change the client default verify option to verify_peer.
    Note that this makes it mandatory to also supply trusted CA certificates or explicitly set verify to verify_none. This also applies when using the so called anonymous test cipher suites defined in TLS versions pre TLS-1.3.

  • Support for Kernel TLS (kTLS), has been added to the SSL application, for TLS distribution (-proto_dist inet_tls), the SSL option {ktls, true}.

  • Improved error checking and handling of ssl options.

  • Mitigate memory usage from large certificate chains by lowering the maximum handshake size. This should not effect the common cases, if needed it can be configured to a higher value.

  • For security reasons the SHA1 and DSA algorithms are no longer among the default values.

  • Add encoding and decoding of use_srtp hello extension to facilitate for DTLS users to implement SRTP functionality.

For more details about new features and potential incompatibilities see the readme

Go to our Download site for the latest versions.
Or to GitHub for the source

Many thanks to all contributors!

29 Likes
3

As a related thing: Rebar3 3.21.0 with support for OTP-26 is now out, including fully handling Dialyzer’s new incremental mode (which we make even more incremental and it’s pretty dang fast!):

12 Likes
4

I’m getting the following dialyzer warning when I use dialyzer on my Advent of Code codebase with Erlang 26:

root@330d0e64d058:/workspaces/advent-of-code/aoc2022# rebar3 do version, dialyzer --incremental
rebar 3.22.0 on Erlang/OTP 26 Erts 14.0
===> Verifying dependencies...
===> Analyzing applications...
===> Compiling aoc2022
===> Dialyzer starting, this may take a while...
===> Running incremental analysis...
===> Resolving project files...
===> Resolving project warning files...

src/day01.erl
Line 0: Unknown function eunit:test/1

The cut down code for day01.erl is:

-module(day01).

-include_lib("eunit/include/eunit.hrl").

%% @doc Highest number of calories carried by any one Elf.
p1(Input) ->
    hd(counts(Input)).

%% @doc Calculate Calories per Elf and sort in ascending order.
counts(Input) ->
    lists:reverse(
        % Iterate Elves and sort.
        lists:sort([
            % Accumulate snacks per Elf.
            lists:sum([
                binary_to_integer(N)
             || N <- binary:split(Group, <<"\n">>, [global, trim])
            ])
         || Group <- binary:split(Input, <<"\n\n">>, [global, trim])
        ])
    ).

%% @doc Generator for unit tests.
gen_test_() ->
    Prefix = filename:join(["priv", atom_to_list(?MODULE)]),
    {ok, Test} = file:read_file(Prefix ++ "_test_input.txt"),
    [
        ?_assert(p1(Test) =:= 24000)
    ].

I get the same without the --incremental, and after changing the test generator to a standard eunit test function.

1 Like
5

We got the same problem and removed the as test from rebar3 as test dialyzer

That works for now, as we only include eunit in the test profile.

2 Likes
6

Understood. For this particular (albeit possibly niche) use case, I do want to run dialyzer over eunit test code. Perhaps eunit used to be in the default PLT and this is no longer the case?

1 Like
7

To be specific, the issue was always there. However, starting with OTP-26, the unknown warning type is on by default. This will make Dialyzer complain about any types for which it has no definitions, or for which it can’t infer one, generally because the app isn’t in the PLT.

This is something we knew was a problem but we didn’t want to turn it on in rebar3 and then cause people a lot of grief about minor versions complaining of different stuff. As it turns out, the OTP team turned it on and I was glad to leave the new default in place.

I’ve added support to the no_unknown option for warnings in Dialyzer in Rebar3 however. This brings back the old behavior if you can’t do a proper fix, which would be what a previous poster mentioned: adding the application to your analysis. Adding the following to your config should work:

{dialyzer, [
   % to go back to the old settings
   {warnings, [no_unknown]}, 
   % or alternatively to add eunit to the analysis set
   % without making it a runtime dep
   {plt_extra_apps, [eunit]}
   % to reduce the analysis time, rebar3 constraints the set
   % of applications analyzed by default by using the 'top_level_deps'
   % option here. If you however get errors about transitive deps you
   % may be calling directly, you can also use the 'all_deps' option
   {plt_apps, all_deps} % default: top_level_deps
}].
4 Likes
8

This was the magic rune I was looking for! Thanks.

1 Like
9

Another thing I’ve noticed that perhaps could be improved. In the Erlang Docker container, rebar3 is installed to /usr/local/bin; however, this is substantially slower to run than when installed with rebar3 do local install, local upgrade (with .cache/rebar3/bin prefixed to the path). Is this something that could be done in the default Erlang Dockerfile?

NB: I got the issue below when trying to test this using the erlang:26-slim image. Fine with vanilla erlang:26.

> [dev_container_auto_added_stage_label 2/4] RUN DIAGNOSTIC=1 rebar3 do local i
nstall, local upgrade:
#0 1.059 ===> Expanded command sequence to be run: [do]
#0 1.059 ===> Running provider: do
#0 1.069 ===> Expanded command sequence to be run: [{local,install}]
#0 1.074 ===> Running provider: {local,install}
#0 1.114 ===> Extracting rebar3 libs to /root/.cache/rebar3/vsns/3.20.0/lib...
#0 1.197 ===> Writing rebar3 run script /root/.cache/rebar3/bin/rebar3...
#0 1.197 ===> Add to $PATH for use: export PATH=/root/.cache/rebar3/bin:$PATH
#0 1.197 ===> Expanded command sequence to be run: [{local,upgrade}]
#0 1.197 ===> Running provider: {local,upgrade}
#0 1.290 ===> Uncaught error in rebar_core. Run with DIAGNOSTIC=1 to see stacktr
ace or consult rebar3.crashdump
#0 1.290 ===> Uncaught error: {badmatch,{error,enoent}}
#0 1.290 ===> Stack trace to the error location:
#0 1.290 [{pubkey_os_cacerts,get,0,[{file,"pubkey_os_cacerts.erl"},{line,38}]},
#0 1.290  {httpc,ssl_verify_host_options,1,[{file,"httpc.erl"},{line,476}]},
#0 1.290  {httpc,http_options_default,0,[{file,"httpc.erl"},{line,1012}]},
#0 1.290  {httpc,http_options,1,[{file,"httpc.erl"},{line,927}]},
#0 1.290  {httpc,handle_request,9,[{file,"httpc.erl"},{line,771}]},
#0 1.290  {rebar_prv_local_upgrade,request_online,2,
#0 1.290                           [{file,"/usr/src/rebar3-src/apps/rebar/src/re
bar_prv_local_upgrade.erl"},
#0 1.290                            {line,138}]},
#0 1.290  {rebar_prv_local_upgrade,maybe_fetch_rebar3,1,
#0 1.290                           [{file,"/usr/src/rebar3-src/apps/rebar/src/re
bar_prv_local_upgrade.erl"},
#0 1.290                            {line,93}]},
#0 1.290  {rebar_prv_local_upgrade,do,1,
#0 1.290                           [{file,"/usr/src/rebar3-src/apps/rebar/src/re
[2023-05-26T10:39:20.893Z] 
bar_prv_local_upgrade.erl"},
#0 1.290                            {line,53}]}]
#0 1.290 ===> When submitting a bug report, please include the output of `rebar3
 report "your command"`
1 Like
10

In OTP 26 it looks like a bug that caused catch expressions in tail position to be optimized away was fixed. For example

foo() ->
    {'EXIT', {bar, _}} = catch error(bar).

wouldn’t actually return {'EXIT', {bar, _}}. It would result in an error, as if it had been written

foo() ->
    error(bar).

Can anyone point me to the pull request that fixed this or any issues related to it? I’m just curious about how it was fixed.

2 Likes
11

I finally got round to upgrading a couple of machines from 25 to 26 and tried out all the shell updates. Major quality-of-life improvements, the autocomplete is brilliant but even the ^L is just such a relief. Thanks so much for all the work that went into these!

5 Likes