Testing Erlang/OTP built from source

Questions / Help
,
1

Hi everyone!

I have been trying to build Erlang/OTP from source according to the installation instructions.
While the build is successful and functional, I am running into some confusion with running the tests. The installation instructions suggest running the smoke tests via make, which complete successfully.
However, when trying to run all test suites (for example by running make test as explained in the testing howto) on an Ubuntu 22.04 machine, some test failures pop up, with results differing based on the OTP version.

What I’m wondering is if there is any status on tests that are expected to fail on a certain OTP version/OS or whether they might be dependent on some kind of configuration? The only related discussion I have encountered is this question from 2017 on the erlang-questions mailing list.

Sorry if I’m completely off the mark here, or the complete test suites are considered an implementation detail that is relevant specifically for development.
Feel free to share if you have run into similar issues in the past, too. Cheers!

3 Likes
2

There are some tests that can fail due to configuration issues and some that can fail due to race conditions. The goal is of course make all tests pass on all platforms all the time, but time is a limiting factor here. If configured correctly, 100% of the tests should pass most of the time on Ubuntu 22.04.

There is a limited set of tests that can be run just to verify that the compilation of Erlang/OTP worked as extected called “smoke test”. You can read about how to run those here: Building and Installing Erlang/OTP — Erlang System Documentation v27.0

If you want to get an idea of what configuration is needed to make all tests pass the best place to look is the docker files used by github actions. They are located here: otp/.github/dockerfiles at master · erlang/otp · GitHub

We do not have a list of which testcases have issues with configuration or racyness, so if you want to have a clean test run then what you need to do is to check each testcase failure and see if it is racyness or configuration.

4 Likes