Erlang/OTP 27.0-rc1 Released

OTP 27.0-rc1

Erlang/OTP 27.0-rc1 is the first release candidate of three before the OTP 27.0 release.

The intention with this release is to get feedback from our users.
All feedback is welcome, even if it is only to say that it works for you.
We encourage users to try it out and give us feedback either by creating an issue at https://github.com/erlang/otp/issues or by posting to Erlang Forums.

All artifacts for the release can be downloaded from the Erlang/OTP Github release and you can view the new documentation at https://erlang.org/documentation/doc-15.0-rc1/doc. You can also install the latest release using kerl like this:

kerl build 27.0-rc1 27.0-rc1.

Erlang/OTP 27 is a new major release with new features, improvements as well as a few incompatibilities. Some of the new features are highlighted below.

Many thanks to all contributors!

Highlights

Documentation

EEP-59 has been implemented. Documentation attributes in source files can now be used to document functions, types, callbacks, and modules.

The entire Erlang/OTP documentation is now using the new documentation system.

New language features

  • Triple-Quoted Strings has been implemented as per EEP 64 to allow a string to encompass a complete paragraph.

  • Adjacent string literals without intervening white space is now a syntax error, to avoid possible confusion with triple-quoted strings.

  • Sigils on string literals (both ordinary and triple-quoted) have been implemented as per
    EEP 66. For example, ~"Björn" or ~b"Björn" are now equivalent to <<"Björn"/utf8>>.

Compiler and JIT improvements

  • The compiler will now merge consecutive updates of the same record.

  • Safe destructive update of tuples has been implemented in the compiler and runtime system. This allows the VM to update tuples in-place when it is safe to do so, thus improving performance by doing less copying but also by producing less garbage.

  • The maybe expression is now enabled by default, eliminating the need for enabling the maybe_expr feature.

  • Native coverage support has been implemented in the JIT. It will automatically be used by the cover tool to reduce the execution overhead when running cover-compiled code. There are also new APIs to support native coverage without using the cover tool.

  • The compiler will now raise a warning when updating record/map literals to catch a common mistake. For example, the compiler will now emit a warning for #r{a=1}#r{b=2}.

ERTS

  • The erl command now supports the -S flag, which is similar to the -run flag, but with some of the rough edges filed off.

  • By default, escripts will now be compiled instead of interpreted. That means that the compiler application must be installed.

  • The default process limit has been raised to 1048576 processes.

  • The erlang:system_monitor/2 functionality is now able to monitor long message queues in the system.

  • The obsolete and undocumented support for opening a port to an external resource by passing an atom (or a string) as first argument to open_port(), implemented by the vanilla driver,
    has been removed. This feature has been scheduled for removal in OTP 27 since the release of OTP 26.

  • The pid field has been removed from erlang:fun_info/1,2.

  • Multiple trace sessions are now supported.

STDLIB

  • Several new functions that accept funs have been added to module timer.

  • The functions is_equal/2, map/2, and filtermap/2 have been added to the modules sets, ordsets, and gb_sets.

  • There are new efficient ets traversal functions with guaranteed atomicity. For example, ets:next/2 followed by ets:lookup/2 can now be replaced with ets:next_lookup/1.

  • The new function ets:update_element/4 is similar to ets:update_element/3, but takes a default tuple as the fourth argument, which will be inserted if no previous record with that key exists.

  • binary:replace/3,4 now supports using a fun for supplying the replacement binary.

  • The new function proc_lib:set_label/1 can be used to add a descriptive term to any process that does not have a registered name. The name will be shown by tools such as c:i/0 and observer, and it will be included in crash reports produced by processes using gen_server, gen_statem, gen_event, and gen_fsm.

  • Added functions to retrieve the next higher or lower key/element from gb_trees and gb_sets, as well as returning iterators that start at given keys/elements.

common_test

  • Calls to ct:capture_start/0 and ct:capture_stop/0 are now synchronous to ensure that all output is captured.

  • The default CSS will now include a basic dark mode handling if it is preferred by the browser.

crypto

  • The functions crypto_dyn_iv_init/3 and crypto_dyn_iv_update/3 that were marked as deprecated in Erlang/OTP 25 have been removed.

dialyzer

  • The --gui option for Dialyzer has been removed.

ssl

  • The ssl client can negotiate and handle certificate status request (OCSP stapling support on the client side).

tools

  • There is a new tool tprof, which combines the functionality of eprof and cprof under one interface. It also adds heap profiling.

xmerl

  • As an alternative to xmerl_xml, a new export module xmerl_xml_indent that provides out-of-the box indented output has been added.

For more details about new features and potential incompatibilities see the README.

18 Likes

Will the documentation search stay as it currently is or will (current) Algolia search be integrated on top of that?

1 Like

It will not remain as it currently is. Either algolia will be integrated, or we will use the planned ExDoc cross repo search.

4 Likes

@AstonJ Maybe that topic should be pinned?

1 Like

EEP-59 ends with a suggestion of doctests being a possibility.
Is this already planned for OTP 27 or is it going to be in a future release?

There will be no doctests support in OTP 27. We have not yet decided if we will work on it for 28. As always, if it is a feature that you would want, a PR would be most welcome.

2 Likes

Installing with kerl as part of building a Docker container, I get warnings about the lack of ex_doc/elixir. Is there a straightforward fix/workaround for this? It’s not a big deal for me as I use online documentation, but since documentation is one of the feature highlights for OTP 27, it’d be good to know how to build it.

#7 42.72 APPLICATIONS DISABLED (See: /root/.kerl/builds/27.0-rc1/otp_build_27.0-rc1.log)
#7 42.73 DOCUMENTATION INFORMATION (See: /root/.kerl/builds/27.0-rc1/otp_build_27.0-rc1.log)
#7 42.73  * documentation  : 
#7 42.73  *                  ex_doc is missing.
#7 42.73  *                  elixir is missing.
#7 42.73  *                  The documentation cannot be built.
1 Like

When you build the HTML docs, you will be asked if you want to automatically download ExDoc and if you say yes, then all the docs can be built. Elixir is not needed to build the docs, but rather for verifying that documentation updates are correct. So it should be be there at all.

So in other words, that warning is a bit misleading, I will make it better or remove it alltogether.

1 Like

Probably not entirely unrelated, but rebar3 ex_doc now fails in my container with 27.0 RC1 vs 26.x. It also seems to want elixir installed. On wrong computer so don’t have a trace to post, but can do so if required. Is this expected behaviour?

The ex_doc plugin should, as far as I know, include ExDoc in it, so there should be no need for elixir before or after Erlang/OTP 27. So I think this is not expected.

If you can provide a small example we can have a look and see if it is Erlang or rebar3 ex_doc that is doing something strange.

1 Like

rebar3 ex_doc output, using a default project created with rebar3 new umbrella foo:

root@17a90bcb498e:/workspaces/ex_doc_test# rebar3 ex_doc
===> Fetching rebar3_ex_doc v0.2.21
===> mv: "mv: failed to preserve ownership for '/workspaces/ex_doc_test/_build/default/plugins/rebar3_ex_doc/CHANGELOG.md': Permission denied\nmv: failed to preserve ownership for '/workspaces/ex_doc_test/_build/default/plugins/rebar3_ex_doc/LICENSE.md': Permission denied\nmv: failed to preserve ownership for '/workspaces/ex_doc_test/_build/default/plugins/rebar3_ex_doc/README.md': Permission denied\nmv: failed to preserve ownership for '/workspaces/ex_doc_test/_build/default/plugins/rebar3_ex_doc/priv/ex_doc': Permission denied\nmv: failed to preserve ownership for '/workspaces/ex_doc_test/_build/default/plugins/rebar3_ex_doc/priv': Permission denied\nmv: failed to preserve ownership for '/workspaces/ex_doc_test/_build/default/plugins/rebar3_ex_doc/rebar.config': Permission denied\nmv: failed to preserve ownership for '/workspaces/ex_doc_test/_build/default/plugins/rebar3_ex_doc/rebar.lock': Permission denied\nmv: failed to preserve ownership for '/workspaces/ex_doc_test/_build/default/plugins/rebar3_ex_doc/src/rebar3_ex_doc.app.src': Permission denied\nmv: failed to preserve ownership for '/workspaces/ex_doc_test/_build/default/plugins/rebar3_ex_doc/src/rebar3_ex_doc.erl': Permission denied\nmv: failed to preserve ownership for '/workspaces/ex_doc_test/_build/default/plugins/rebar3_ex_doc/src': Permission denied\nmv: failed to preserve ownership for '/workspaces/ex_doc_test/_build/default/plugins/rebar3_ex_doc/hex_metadata.config': Permission denied\nmv: failed to preserve ownership for '/workspaces/ex_doc_test/_build/default/plugins/rebar3_ex_doc': Permission denied\n"
===> Analyzing applications...
===> Compiling rebar3_ex_doc
===> Verifying dependencies...
===> Analyzing applications...
===> Compiling foo
===> Running edoc for foo
===> Running ex_doc for foo
===> ERROR! Failed to start Elixir.
error: {error,
           {elixir,
               {bad_return,
                   {{elixir,start,[normal,[]]},
                    {'EXIT',
                        {undef,
                            [{elixir,start,[normal,[]],[]},
                             {application_master,start_it_old,4,
                                 [{file,"application_master.erl"},
                                  {line,295}]}]}}}}}}

===> 

rebar3.config:

{plugins, [rebar3_ex_doc]}.

Dockerfile:

FROM ubuntu:22.04

ARG OTP_VERSION="27.0-rc1"

LABEL otp_version=$OTP_VERSION
LABEL description="Erlang docker-based development environment"

RUN set -xe \
    && runtimeDeps='libodbc1 \
            libsctp1 \
            libwxgtk3.0 \
            libwxgtk-webview3.0-gtk3-0v5' \
    && buildDeps='unixodbc-dev \
            libsctp-dev \
            libwxgtk-webview3.0-gtk3-dev' \
    && apt-get update \
    && apt-get install -y --no-install-recommends $runtimeDeps \
    && apt-get install -y --no-install-recommends $buildDeps \
    && apt-get install -y clang curl git make libssl-dev libncurses-dev \
    && apt-get autoremove \
    && apt-get clean

WORKDIR /root

# Use kerl (https://github.com/kerl/kerl) to install specified version of Erlang.
RUN set -xe \
    && curl -O https://raw.githubusercontent.com/kerl/kerl/master/kerl \
    && chmod a+x kerl \
    && mv kerl /usr/local/bin/ \
    && kerl update releases \
    && kerl build ${OTP_VERSION} \
    && kerl install ${OTP_VERSION} /opt/kerl/${OTP_VERSION} \
    && kerl cleanup all \
    && echo ". /opt/kerl/${OTP_VERSION}/activate" >> ~/.bash_profile

CMD [ ".", "/opt/kerl/${OTP_VERSION}/activate" ]

# Install lastest stable build of rebar3 build tool (https://github.com/erlang/rebar3)
RUN set -xe \
    && curl -O https://s3.amazonaws.com/rebar3/rebar3 && chmod a+x rebar3 \
    && . /opt/kerl/${OTP_VERSION}/activate \
    && ./rebar3 local install \
    && rm rebar3

ENV PATH="/opt/kerl/${OTP_VERSION}/.cache/rebar3/bin:$PATH" \
    ERL_AFLAGS="+pc unicode -kernel shell_history enabled"

I’ve started a few pull requests in tools that are affected by this release:

1 Like

Building locally, with the latest kerl, and KERL_BUILD_DOCS=yes, I get

 * documentation  :
 *                  ex_doc is missing.
 *                  The documentation cannot be built.

Building docs...
Erlang/OTP 27.0-rc1 (27.0-rc1) has been successfully built

but I’m not asked about installing ex_doc locally :confused:

In configure.ac I do see

AC_CHECK_PROGS(EX_DOC, ex_doc)
if test -z "$EX_DOC"; then
  EX_DOC="$ERL_TOP/make/ex_doc_wrapper"
  echo "ex_doc" >> doc/CONF_INFO
  AC_MSG_WARN([No 'ex_doc' command found: the documentation cannot be built])
fi

and ex_doc_wrapper would eventually install ex_doc but I don’t see it invoked :confused:


Later I tried to invoke . ./make/ex_doc_wrapper directly, on macOS, but run into issues… for example read -p "Do you want to download latest ex_doc from github? (y/n)? " -n 1 -r returns read: -p: no coprocess

@garazdawi, would it change your initial intentions much if you did

-    read -p "Do you want to download latest ex_doc from github? (y/n)? " -n 1 -r
-    echo
+    echo -n "Do you want to download latest ex_doc from github? (y/n)? "
+    read REPLY

and

-        read -p "Press any key to continue..." -n 1 -r
+        echo "Press any key to continue..."
+        read

Later I also saw the documentation mentions

In order to build the documentation you need to have ex_doc
installed and available in your path. The simplest way to do that is to download
the escript for the latest release available on github.

so maybe the “wrapper” is not supposed to be invoked directly (as of now) and is just a helper. (?) Please let me know (link to kerl issue. Update to allow generating documentation from OTP 27 on · Issue #509 · kerl/kerl · GitHub) if we can do something automatically, with kerl, to ease building the documentation.

2 Likes

I opened a rebar3_ex_doc issue to track this (Update to allow generating documentation from OTP 27 on · Issue #509 · kerl/kerl · GitHub), which is replicated by a PR (Test for OTP matrix by paulo-ferraz-oliveira · Pull Request #80 · starbelly/rebar3_ex_doc · GitHub). Feel free to discuss there, too, since this seems more plugin-specific than OTP-specific.

Edit: it seems, at this moment, we only need a rebar3_ex_doc release and the issue should be gone :cloud:. I updated CI so as to not use Hex’s rebar3_ex_doc (but the very version we build), and it shows no errors. I didn’t dig into why this is, but it seems something changed in ex_doc from 0.30.7 to 0.31.1 that made it so :smile:

2 Likes

I can confirm that rebar3 ex_doc no longer complains if I revert the version of rebar3_ex_doc plugin to 0.2.12.

rebar.config:

{plugins, [{rebar3_ex_doc, "0.2.12"}]}.

I think the reason why it fails is because the bundled ExDoc is built using Erlang/OTP 24 and not all .beam files produced by the 24 compiler can be loaded by the 27 runtime system. This is not related to the usage of ExDoc in Erlang/OTP, but would have happened no matter.

ExDoc provised pre-built binaries for 24+25+26, maybe it would be a good idea for the rebar3 plugin to bundle more versions and use different ones depending on which version of Erlang is running?

3 Likes

Not a bad idea, @starbelly :smile:

3 Likes

I think that’s a solid idea :smile:

1 Like

Built 27.0-rc1 and took it for a spin - mainly trying to shake out what dependenices needs updating etc. So far no major surprises :+1:

But when I build a release using rebar3/relx ({include_erts, true}) it seems to include wx in the resulting artifact, despite not being mentioned anywhere. This does not happen with OTP22-26?!

Also tried explicitly excluding wx from the build with --without-wx but that fails miserably with:

...

*********************************************************************
**********************  APPLICATIONS DISABLED  **********************
*********************************************************************

jinterface     : Java compiler disabled by user
odbc           : ODBC library - link check failed
wx             : User gave --without-wx option

...

 ERLC	../ebin/dbg_wx_trace.beam
compile: warnings being treated as errors
dbg_wx_filedialog_win.erl:23:2: behaviour wx_object undefined
%   23| -behaviour(wx_object).
%     |  ^

make[3]: *** [../ebin/dbg_wx_filedialog_win.beam] Error 1
make[3]: *** Waiting for unfinished jobs....
make[2]: *** [opt] Error 2
make[1]: *** [opt] Error 2
make: *** [libs] Error 2

Edit:

According to Issue 7694 you need --without-wx --without-debugger --without-observer --without-et to disable wx :man_shrugging:

Edit2:
I think the culprit is observer being a dependency (of a dependency) and in OTP-27 it seems like observer have a new entry {optional_applications, [wx, ...]} could this mean that wx is all of a sudden included in a release?