Odd asdf install behaviour: ERROR: 'asdf_27.2' is not a kerl-managed Erlang/OTP installation / Waiting for unfinished jobs

AstonJ · January 7, 2025, 12:27am

Installing Erlang using asdf install erlang latest and kept getting ERROR: build failed even after trying things like brew install kerl and brew install ncurses-devel - thought I’d try restarting the machine and it worked fine after that so am just logging this here in case anyone else comes across it in future.

Output before reboot:

ERROR: 'asdf_27.2' is not a kerl-managed Erlang/OTP installation.
Build 'asdf_27.2' has been deleted.
Extracting source code for normal build...
Building (normal) Erlang/OTP 27.2 (asdf_27.2); please wait...
Initializing (build) log file at /Users/Aston/.asdf/plugins/erlang/kerl-home/builds/asdf_27.2/otp_build_27.2.log.
Attempting to use Homebrew OpenSSL from /opt/homebrew/opt/openssl@3.0...
WARNING: ... you may have to brew the expected version or otherwise use --with-ssl
APPLICATIONS DISABLED (See: /Users/Aston/.asdf/plugins/erlang/kerl-home/builds/asdf_27.2/otp_build_27.2.log)
 * jinterface     : No Java compiler found
 * odbc           : ODBC library - link check failed

APPLICATIONS INFORMATION (See: /Users/Aston/.asdf/plugins/erlang/kerl-home/builds/asdf_27.2/otp_build_27.2.log)
 * wx             : wxWidgets was not compiled with --enable-webview or wxWebView developer package is not installed, wxWebView will NOT be available
 *         wxWidgets must be installed on your system.
 *         Please check that wx-config is in path, the directory
 *         where wxWidgets libraries are installed (returned by
 *         'wx-config --libs' or 'wx-config --static --libs' command)
 *         is in LD_LIBRARY_PATH or equivalent variable and
 *         wxWidgets version is 3.0.2 or above.

ERROR: build failed.
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/unistd.h:639:10: fatal error: cannot open file '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/sys/select.h': Too many open files
  639 | #include <sys/select.h>
      |          ^
1 error generated.
make[4]: *** [obj/aarch64-apple-darwin24.1.0/opt/jit/beam_asm_module.o] Error 1
make[4]: *** Waiting for unfinished jobs....
make[3]: *** [opt] Error 2
make[2]: *** [opt] Error 2
make[1]: *** [jit] Error 2
make: *** [emulator] Error 2

Please see /Users/Aston/.asdf/plugins/erlang/kerl-home/builds/asdf_27.2/otp_build_27.2.log for full details.
Auto cleaning all artifacts except the log file...
(use KERL_AUTOCLEAN=0 to keep build on failure, if desired)
Cleaning up compilation products for 'asdf_27.2' under:
  - /Users/Aston/.asdf/plugins/erlang/kerl-home/builds...
  - /Users/Aston/.asdf/downloads/erlang/27.2...
... done.

Output after reboot:

(Everything looks the same apart from what’s below the lines ERROR: build failed from above and Erlang/OTP 27.2 (asdf_27.2) has been successfully built in the below)

ERROR: 'asdf_27.2' is not a kerl-managed Erlang/OTP installation.
Build 'asdf_27.2' has been deleted.
Extracting source code for normal build...
Building (normal) Erlang/OTP 27.2 (asdf_27.2); please wait...
Initializing (build) log file at /Users/Aston/.asdf/plugins/erlang/kerl-home/builds/asdf_27.2/otp_build_27.2.log.
Attempting to use Homebrew OpenSSL from /opt/homebrew/opt/openssl@3.0...
WARNING: ... you may have to brew the expected version or otherwise use --with-ssl
APPLICATIONS DISABLED (See: /Users/Aston/.asdf/plugins/erlang/kerl-home/builds/asdf_27.2/otp_build_27.2.log)
 * jinterface     : No Java compiler found
 * odbc           : ODBC library - link check failed

APPLICATIONS INFORMATION (See: /Users/Aston/.asdf/plugins/erlang/kerl-home/builds/asdf_27.2/otp_build_27.2.log)
 * wx             : wxWidgets was not compiled with --enable-webview or wxWebView developer package is not installed, wxWebView will NOT be available
 *         wxWidgets must be installed on your system.
 *         Please check that wx-config is in path, the directory
 *         where wxWidgets libraries are installed (returned by
 *         'wx-config --libs' or 'wx-config --static --libs' command)
 *         is in LD_LIBRARY_PATH or equivalent variable and
 *         wxWidgets version is 3.0.2 or above.

Erlang/OTP 27.2 (asdf_27.2) has been successfully built.
Cleaning up compilation products for 'asdf_27.2' under:
  - /Users/Aston/.asdf/plugins/erlang/kerl-home/builds...
  - /Users/Aston/.asdf/downloads/erlang/27.2...
... done.

(So you might just need to reboot your machine)

starbelly · January 9, 2025, 11:48am

This seems like it is mostly for information purposes, but let me see if I can shed a little bit more light on what you encountered. I have seen this happen previously a few times, just never on my machine (see The mess we’re in by Joe). It is is always in relation to asdf (though this is just correlation).

If we look closely at when the build failed we can see the trunk of the problem :

ERROR: build failed.
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/unistd.h:639:10: fatal error: cannot open file '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/sys/select.h': Too many open files
 639 | #include <sys/select.h>

Mac os ships with a very low user limit on the number of files that can be open at a single point in time (256 I believe). As such, in a session (terminal), if the build, kerl, asdf-erlang, asdf itself, or some other process(s) (or all of the above) in the background opened more than 256 files, well, game over

I usually advise “power” users to set a higher limit on a per session basis via their shell initialization files (e.g., .bash_profile, .zsh_profile, etc.). Just drop ulimit -n 1024 into a shell initialization file (or adjust per your needs) and you should be right as rain

“Unable to find local grunt” - but I just installed grunt - Aaaaaaggggg -
having a bad day … – Joe Armstrong

AstonJ · January 9, 2025, 8:51pm

Thanks Bryan! I think you might be right!

I originally thought the same, however on checking it reported:

$ ulimit
unlimited

$ ulimit -n
2560

So I assumed it wasn’t that, however now if I run launchctl limit I get the following (the above is the same):

$ launchctl limit
	cpu         unlimited      unlimited      
	filesize    unlimited      unlimited      
	data        unlimited      unlimited      
	stack       8372224        67092480       
	core        0              unlimited      
	rss         unlimited      unlimited      
	memlock     unlimited      unlimited      
	maxproc     2666           4000           
	maxfiles    256            unlimited

Wonder why there’s a disparity between ulimit -n and launchctl limit?

Oddly, running the same on an Intel Mac which I’ve not had problems with when installing Erlang I get:

$ ulimit
unlimited

$ ulimit -n
256

$ launchctl limit
	cpu         unlimited      unlimited      
	filesize    unlimited      unlimited      
	data        unlimited      unlimited      
	stack       8388608        67104768       
	core        0              unlimited      
	rss         unlimited      unlimited      
	memlock     unlimited      unlimited      
	maxproc     11136          16704           
	maxfiles    256            unlimited

eproxus · January 14, 2025, 9:47am

I haven’t had this exact problem (I use mise which now install EEF pre-built Erlang installations on macOS) but here are my notes for how to permanently increase the file ulimits on macOS in case you want it for all users/shells/etc:

How To Increase macOS Kernel maxfiles (ulimit)

Check the limit by running:

$ sudo launchctl limit
$ ulimit -a

Create /Library/LaunchDaemons/limit.maxfiles.plist:

$ sudo vim /Library/LaunchDaemons/limit.maxfiles.plist

Paste the following contents:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"      "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Label</key>
    <string>limit.maxfiles</string>
    <key>ProgramArguments</key>
    <array>
      <string>launchctl</string>
      <string>limit</string>
      <string>maxfiles</string>
      <string>1048576</string>
      <string>1048576</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>ServiceIPC</key>
    <false/>
  </dict>
</plist>

Run sudo chown root:wheel /Library/LaunchDaemons/limit.maxfiles.plist
Run sudo launchctl load -w /Library/LaunchDaemons/limit.maxfiles.plist
Restart the machine to reflect the settings in ulimit -a