@mmin good shout with DEBUG=1. It looks like the issue was with caching the resulting .erl files generated from several large leex/yecc files. Saving those seems to have done the trick, an easy oversight. It is hard to generalize this as a single solution for future build-cachers.
- If you’re using git, set the mtime of the source files to the latest commit. This allows for new commits to rebuild the correct files. I relied on this script.
- If you’re using
leex/yecc, cache the generated.erlfiles. - Avoid compile-time macros, defined using
-dto the erl compiler, that will change between builds. Especially if they’re resolved in a large header file chain. - Try to compile the deps first? Not sure if this one matters all that much.
- If you’re caching your build in cloud storage like S3, try and tar + gzip or xz the
_builddirectory, it saved a ton of time compared to downloading the individual files (30 minutes vs 5 seconds in my case).- Also don’t rely on your build system’s per-file caching. The one I was using modified the mtime of individual artifacts, which seems to have been triggering some builds. tar them all manually, which will preserve mtime.
- If you’re using NIFs, it doesn’t appear as though caching the
.sofiles matters all that much if they’re very simple, YMMV. - This scheme works well for ‘fresh builds’, for example I am not going to untar the artifact on the
mainbranch.