The EEF is looking for volunteers to take over ESL's build packages

Chat / Discussions
1

The Build and Packaging working group at the EEF has had talks with folks at Erlang Solutions to hand this project off to the foundation, before we accept we’re looking for someone to own the project.

Becoming the owner means at least being accountable to let us know the status of it, be a point of contact for its content, and being able to triage issues and whatnot. The expectation isn’t necessarily to maintain everything, but just to be able to answer questions and provide vision for it.

If you’d like to help please let us know!

8 Likes
2

From the perspective of the EEF/working group, it makes sense to first ask for a owner who will then find people to take on the specific tasks, ie to fill the spots in an as yet empty shell.

What you ask for is somebody to manage an unscheduled promise. But if the other people around here are remotely like me, I guess you won’t find a match easily. I am a freelancer. I can pick the tasks I take, but if I take a task it implies a strong promise to deliver something tangible, within a deadline.

So… i think what the EEF working group needs to decide on is…

  • For what OSes should packages be provided;
    that doesn’t have to be all at once and fixed forever, more like a priorities-and-wishes list;
    as people were asking mostly about packages for Ubuntu jammy, that should rank high
  • What range of OTP releases should packages be made available;
    ESL provides packages down to R13 for some OSes, that isn IMO overkill;
    the latest versions of the latest minor releases of the actively supported releases are IMO enough;
    the latest versions of the latest major releases of the actively supported releases are probably enough

This, in turn, determines what kind of people you are looking for, like “Hey, I know how to do packaging for this OS (but not for that other OS)” (and if you dear reader are one of those people, raise your hand please!!!) Preferably, you can find several of each, bus factor and all.

Apart from that, you will need somebody to automate all this, mabye via GitHub Actions other something.

Lastly, you will need some place to host it all. This again I see in the hands of the EEF/working group, since it requires a long-term commitment from some company or other. the requirements, for space mostly, is determined by how many OSes (now, and later) and OTP releases (now and later) are to be supported.

Put differenty, you are asking for an engineer to asemble a clock. Among us are the gears that can be assembled into a clockwork, but looking for an engineer.

2 Likes
3

Yep, those are good points. I do not use the ESL packages and I have no idea what they are committed to.

However we have been told that:

  • Hosting was mostly solved (and the foundation would handle the finances)
  • Everything is automated and based on scripts
  • The only action needed is often to patch up scripts and new builds when something breaks on a new release

The sort of understanding I had is that we wouldn’t necessarily expect the “owner” to write all the fixes – we’re hoping the community would help write them as things break – but to be the sort of person who can review and shepherd them, and report about the overall project health to the working group.

Bigger drives (say the infrastructure is outdated and needs love) are probably things that could be surfaced by that owner, but ideally wouldn’t fall on their shoulders to fix. The ideal situation we’d be in is one of minimal “keep it going” efforts, not one of driving major improvements and development to the product unless they felt very passionate about it, in which case we wouldn’t stand in their way.

3 Likes
4

Sounds right up your street @Maria-12648430 :081:

Seriously tho, maybe it’s something more than one person can more easily commit to? Since it seems pretty important to Maria, @mworrell and @juhlig (and others) maybe you could give it a go and if it doesn’t work out the worst that can happen is we’re back to square one?

*runs

5

I tried looking into that some time ago. There are numerous components to build, and some decisions to make. For example, package manager (mechanism): native (rpm, deb) or self-contained (snap)? This also affects release schedule/mechanism (e.g. nightly builds done differently). Snap is easier to begin with, but it may not be available on all platforms. PPA is another relatively easy way to start with.

Also I noticed that ESL packages are somewhat fine-grained, e.g. you can install just ERTS and some separate apps, and skip, say, mnesia or ssh. While it looks useful, in reality I think it does not add much value (I am yet to see anyone using that separation).

If we can agree to a more specific requirements, it would be much easier to onboard a volunteer, or even do some step-by-step work when some free time appears.

3 Likes
6

2 posts were split to a new topic: Erlang packages in distros

7

Yes, same here. So for the future, I would drop this granularity for all-or-nothing.

I agree, if we know better what we are looking for, it will be easier to find people to fill that role :+1:

2 Likes
8

I can come up with a script that does necessary RPM packaging for CentOS. It’s probably not too hard to do Debian’s PPA. Infrastructure can be provided by the EEF (technically it’d be me creating VM/droplet/…).

However I have no experience at all dealing with Debian folks or RedHat maintainers. Hence I have no idea how to include updated packages in the OS repository.

And there are so many other requirements choices.

  1. OTP build options. I’m not sure whether we want defaults, or something more elaborate. For example, I always include “extended” MSACC states (because otherwise catching ETS lock contention is tedious).
  2. Apps to include. Just OTP? All apps? Do I add rebar3? (this is what I’ve done for my distribution anyway)
  3. How to test that, and which distributions to test with? That was the roadblock I hit a year ago. There is no CI for OTP that I can leverage. I attempted to build my own, and still think it’s the way to go. But unfortunately I could not maintain momentum, for my employer apparently does not support this open source work.
2 Likes
9

Maybe start with the popular OS’s? Windows, macOS and maybe Ubuntu? Not only that but limited options regarding apps. Basically, you get everything. Support for multiple OS’s can be done later.

1 Like
10

I’m not sure if it holds for all (as in deb vs rpm) packages. I think it only does for Debian/Ubuntu and is strictly because of Debian packaging policy - Debian strives to split up big packages into small ones so that the user has choice. One of the assumptions when the ESL packages were originally created was that they should be drop-in replacements for, often outdated, distro-specific packages, hence the exact mirror of the Debian package structure. Indeed, in case of Erlang packages it brings a lot of hassle - both for the user, as well as for the maintainer - as the OTP libs have many interdependencies.

AFAICT there’s also a unified package under a different name - esl_erlang - which bundles everything into one .deb, for those people who don’t care about a “drop-in”, but want convenience of a single package.

We’ve been there, too. I think it’s better to only ship whatever OTP ships. Otherwise, the extra bundled libs will conflict with per-project deps and might lead to diamond dependency problems.

1 Like
11

Linux has many distributions with incompatible package management systems.
RabbitMQ team agreed to support Windows already, and for MacOS it’s done via brew (which I think already has some level of support). So we’re mostly talking of Linux - and mainly it’s Debian and RedHat, hence *.deb and *.rpm if we’re disregarding Snap packages.

1 Like
12

Makes sense. asdf works nicely on Mac too. They seem to be on the ball providing the latest version.

1 Like
13

BTW the actual support for Erlang versions that asdfrelys on is kerl:

1 Like
14

Maybe it’s worth sharing the codebase of ESL’s packaging repo here. The readme says it’s “a new approach” so I’m sure if it was ever deployed and how complete it is.

2 Likes
15

@michallepicki I have been looking at that and can’t make neither heads nor tails of it :sweat_smile:

1 Like
16

I will just add that for anyone using Nix and NixOS, we already have people (and bots) doing the patch update. And I will usually do the yearly release. It is relatively easy, but that is due to the nature of Nix.

Nix also supports Mac, so for these fronts, we are covered. I doubt this is portable, but I thought it was worth to point out this does not need more support.

1 Like
17

Hello,

Well, I used to use ESL packages mainly for debian and centos; for production and development purposes. When installing ESL packages I always wanted a minimal dependency set.

I can help. Not alone, but with more people :), it’s a huge task here.

I’m also meta-erlang maintainer (GitHub - meta-erlang/meta-erlang: Erlang and Elixir support, https://meta-erlang.github.io/). There we have the same sort of problem but with another perspective and goals.

3 Likes
18

Yay :partying_face: First volunteer!

No idea if I could be of any help, never done anything like that before, and like everyone else I have not that much time, but if you need a hand, feel free to ask :wink:

Yeah, same here, but for Ubuntu.

2 Likes
19

Not sure if it’s any help but saw this on devtalk:

1 Like
20

ESL has gotten a new DevOps person (@dalbarado) who has started working on the build stuff again and has reached out to us at the EEF about it. If anyone was very interested (@joaohf ?) we can probably put everyone in sync about part of this knowledge transfer.

5 Likes