Ocibuild - Build OCI compliant images from your releases

Hello,

I have been working on a rebar3 plugin/library for the last few days that allows you to build OCI compliant container images from Erlang and Elixir releases. At my company we’re using quite a few different languages, and some of them (C# and Golang are quite popular) have the ability to build container images directly without having to go through Docker. This results in incredibly fast build times on CI/CD pipelines.

I was a bit jealous of this for my Elixir projects, so I looked into if it was possible to do so for BEAM languages. With a lot of help from my good friend Claude we made fast progress, and tonight I released the first v0.1.0 release to Hex.pm.

Disclaimer: I’ve not written much Erlang before, but I decided to write this in Erlang instead of Elixir in order to make it easier for all BEAM languages to take advantage of this.

If this sounds interesting, please give it a spin :slight_smile:

11 Likes

Hi, friends!

During the last few days I’ve implemented quite a few new features from my roadmap for
this library. I just released v0.5.0 which includes the following new features.

  • Multi-Platform Images

    • We can now build images for multiple platforms using a single command.
    • All downloading and uploading of layers now runs in parallel.
    • Multi-arch manifest follows the standard OCI format.
  • Non-Root containers by default

    • Runs as UID 65534 (nobody) by default, can be overridden using the --uid flag.
  • Automatic OCI Annotations

    • Generate OCI labels/annotations automatically from release version and VCS (only tested with Git for now).
  • Reproducable Builds

    • Respects the SOURCE_DATE_EPOCH env variable to override container file timestamps. This allows us to create reproducable builds given the same input.
  • Automatic Software Bill of Materials (SBOM) support

    • SPDX 2.2 SBOM are included in every image.
    • Can also be written to file using the --sbom flag.
  • Smart Dependency Layering

    • ERTS, dependencies, application code and SBOM are written as separate layers, meaning that only changes are pushed to registry. This results in typically 80-90% smaller uploads.

There’s probably still a few rough edges and the code-base needs a bit refactoring in certain places, but I have implemented most of my planned features and would be very happy for feedback (both positive and negative)!

As stated earlier, I’m, quite fresh in Erlang (even though I have been coding Elixir for quite some time), so there might be conventions and other things that could use some polish.

3 Likes

Hello again!

Just a small update on the latest work on this library. I’ve now implemented all planned features and made significant improvements on the code base, both in terms of code organization, test coverage and security measures.

The library now has the following feature table.

Feature Status Description
No Docker required :white_check_mark: Builds images directly without container runtime.
OCI compliant :white_check_mark: Produces standard OCI image layouts.
OCI annotations :white_check_mark: Add custom annotations to image manifests. Automatically populate source URL, revision, version from VCS.
Push to any registry :white_check_mark: Docker Hub, GHCR, ECR, GCR, and any OCI-compliant registry.
Build system integration :white_check_mark: Native rebar3 and Mix task support.
Non-root by default :white_check_mark: Run as non-root (UID 65534) by default; override with --uid.
Layer caching :white_check_mark: Base image layers cached locally for faster rebuilds.
Tarball export :white_check_mark: Export images for podman, skopeo, crane, buildah, etc.
Multi-platform images :white_check_mark: Build for multiple architectures (amd64, arm64) from a single command.
Reproducible builds :white_check_mark: Identical images from identical inputs using SOURCE_DATE_EPOCH.
Smart dependency layering :white_check_mark: Separate layers for ERTS, dependencies, and application code.
SBOM generation :white_check_mark: SPDX 2.2 SBOM embedded at /sbom.spdx.json and attached via referrers.
Image signing :white_check_mark: Sign images with ECDSA keys (cosign-compatible format).
Zstd compression :white_check_mark: Automatic zstd on OTP 28+ (20-50% smaller, faster). Falls back to gzip on OTP 27.

*Disclaimer: The claim about pushing to “any” registry is theoretical. I’ve actually only tested it with ghcr.io.

The library should be on par (or better) feature-wise with comparable libraries from other eco-systems (ko for Golang, Microsoft.NET.Build.Containers for .NET and jib for Java). It will remain in v0.x.x for a while to gather real-world experience and fixing bugs that certainly will pop up, but it should be in a pretty good shape.

Hopefully someone will find it useful :slight_smile:

2 Likes

This looks interesting. Given you‘re not running any container this won’t help with compiling NIFs for the target architecture though I guess. So just like ERTS they would need to be handled out of band. This still looks like a great option for where that‘s not needed.

That’s correct. If you’re building images for another platform you’ll need to either have to cross-compile your NIFs with a toolchain for that platform, or use a custom base image which contains ERTS and your NIFs already in place. The upside is that you’ll only have to build this base image once instead of on every build.

@rhblind Thanks for sharing.

Could you please explain?

Zstd compression :white_check_mark: Automatic zstd on OTP 28+ (20-50% smaller, faster). Falls back to gzip on OTP 27.

zstd support is not available in OTP27 (as far as I know), thus we’re falling back to compressing the image using gzip instead.