What would you like to see in Erlang or the BEAM world?

Further to the thread about static typing - what would you personally like to see most in Erlang or the BEAM world?

  • Static typing?
  • Better performance?
  • Better BEAM interop?
  • More BEAM languages?
  • More libraries or tools? (Which ones?)
  • More books, tutorials, courses or screencasts?
  • Something else?

:robot:

Please note if any interesting discussions/forks result from this thread we may split them into dedicated threads.

6 Likes

In the spirit of keeping this general and to avoid bike shedding (if you will) Iā€™m going to keep my reply as short as possible and limited to Erlang/OTP for the moment. As a lot of what I could say has been covered in other threads or will be covered by others. Some of this is redundant, but I like over-communication :slight_smile:

These are in order of priority :

  1. Presentation, presentation, presentation. The new erlang.org has a been huge step in the right direction, but thereā€™s more to do. On one hand we can say this doesnā€™t matter, but IMHO it does, in regards to adoption anyway. Iā€™d ask others who have good design skills to get involved with contributing to erlang.org to improve the overall look, UX, accessiblity, etc.

  2. Documentation, documentation, documentation. I want to see Erlang/OTP move away from XML to markdown or asciidoc and put the docs in with the source code. Itā€™s never pleasant contributing documentation in Erlang/OTP. Elixir and other languages have spoiled some of us. I personally plan on investing a lot of time in this area, but I have to get clean my plate with some other things before the end of the year. There should also be WAY more example code, guides, and tutorials on erlang.org as well.

  3. Performance! Ludicrous speed! This is so important now, for existing users and wider adoption. I asked @jhogberg recently how far can we take JIT as an example, to which he said the jit work done thus far is base line and basically saying the sky is the limit. Beyond JIT thereā€™s surely places to optimize ERTS and Erlang code all over the place.

  4. A slimmed down and more ergonomic standard lib and better UX in general. I think thereā€™s a lot of simple things that can be done to improve the Erlang standard library. I think thereā€™s a lot to learn from Elixir and other languages in general in this regard. One idea I had was an enum module. Of course, we wouldnā€™t have protocols and such, but simply a module that lets you interface with enumerables might not be a bad thing to have. Thereā€™s also lots of little enhancements we could make to existing modules IMO. I also fancy the idea of building on top of OTP to make a framework that nudges people down a ā€œproperā€ OTP design path. Iā€™m not terribly sure what that looks like without a lot of thinking, but I do like the idea. I used to not like these ideas as I thought the barrier to entry being high was a good thing, and I do think thereā€™s some truth to that, but I think in retrospect itā€™s folly, it doesnā€™t solve any problems, and perhaps can even be seen as gatekeeping. I think thereā€™s a lot more to unpack, but thatā€™ll do for now.

  5. Moar libraries! Thatā€™s a huge conversation and itā€™s been had in a few places on here already, but one thing Iā€™ve seen full on stop adoption is there not being a mature lib for X. It shouldnā€™t matter right? But it does, especially to middle management. This leads into BEAM interop as well. Really it comes down to having better support in project/build tools to easily bring in a dep from any other beam language and have an ergonomic standard syntax which would greatly increase adoption of libraries written in other beam languages by Erlang users. On the flip side of that perhaps is there needs to be more core libraries written in Erlang. Itā€™s simply easier to share without requiring all the hoop jumping required to solve BEAM interop in regards to deps and syntax, but I also canā€™t fault folks for wanting to writes libs in languages that make them happy to write in. However, whatā€™s good for the goose is good for the gander, thatā€™s irrefutable IMO.

  6. 42

Thereā€™s way more I could say, but then it goes into specifics, so I shall refrain for now :slight_smile: Of course, I would be remiss to not comment we can wish all day long for X and Y, but itā€™s going to take action to make these things happen, as is tradition. Thatā€™s a whole thread unto itself.

8 Likes

One more :

A united front. There has been great progress in this area over the past few years, but weā€™re not quite there yet. I still very much feel that thereā€™s little factions that refuse to embrace such a notion and I think this exists on all sides. One hand washes the other, full stop, period. And I will say it yet again : Whatā€™s good for the goose is good for the gander.

4 Likes

the sooner the better

4 Likes

Hmmā€¦ Personally I want more! More! Or rather, I want more stuff in the most commonly used modules, not more modules. Stuff that falls on the ā€œmaybe useful sometimesā€ scale. Like the following, ā€œmissingā€ from the lists module:

take(3, "abcdef") -> "abc".
intercalate("...", ["This", "Is", "Sparta!"]) -> "This...Is...Sparta!".
strip_prefix("hell", "hello world") -> "o world".
strip_suffix("orld", "hello world") -> "hello w".
iterate(fun(X) -> X * 2 end, 1, 7) -> [1, 2, 4, 8, 16, 32, 64].
infix("def", "abcdefgh") -> true.
init("1234567890") -> "123456789".
chunk(4, "abcdefghijkl") -> ["abcd", "efgh", "ijkl"].
find_first(fun(X) -> X rem 2 == 0 end, [1, 3, 5, 6, 7, 9, 10, 11]) -> 6.

Even though I donā€™t use all of them that often, and even though some of them may just be special cases of already existing lists functions, it still feels wrong rewriting the chunk function for the dozenth time. I can appreciate that many might prefer a slim API, but I like to have everything useful just there for whenever I need it, and I can trust itā€™s been properly and efficiently written and tested already.

7 Likes

Yeah, ā€œslimmed downā€ is a bit vague here. Definitely agree thereā€™s things that could be added to improve the standard lib, and I think thereā€™s things that could be removed or in some cases folded into one module.

3 Likes

I have to protest against the phrase ā€œall over the placeā€. To me that sounds like a code base where most of the attention has been spent on implementing new features and making them work, and very little attention on efficiency. I am sure thatā€™s not what you meant.

I would say that there has always been the ambition to make Erlang faster.

As Erlang has matured, it has become increasingly harder to do more meaningful optimizations. All the obvious and easy optimizations have been done. New optimizations either complicates the implementation or depends on a tradeoff (making a frequently used operation faster while making other infrequently used operations slower). Usually both.

That said, we in the OTP team also find efficiency important and will continue to do optimizations. For example, in OTP 25 the compiler will store type information in the BEAM file. The JIT can then generate slightly better code using the type information (for example, remove redundant type tests). While the mechanism for transferring type information to the JIT is introduced in OTP 25, we expect to take more advantage of it in OTP 26. (It is not that much time left for major optimizations in OTP 25, because we want all them to go in to the first release candidate of OTP 25 (released during the first quarter of 2022) to ensure that they get tested by as many people as possible.)

9 Likes

In contrast to what @Bryan Paxton and many others state, I am very happy with what Erlang/OTP is today. For me, the unique selling points are:

  • The language is simple. It keeps me focused on what I want to implement and not on how to use all kinds of language constructs. Do not follow the Python path (or C++, Java, Elixir orā€¦). Keep it simple and effective!
  • I find the Erlang documentation probably the best there is around. Yes, it is dense, concise but also accurate, precise, up-to-date and itā€™s layout actually is very good to read, no clutter. The answer is always there, one just has to read carefully! After learning the language from a proper (non-free) textbook and some serious efforts to understand the language / framework, the documentation is more than adequate.
  • It is not about the presentation! This is an engineering tool and I donā€™t want clutter. Take Googleā€™s search engine interface and UX. Keep it simple! For example, I hate the new search box on the left in the Erlang documentation which makes this pop-up appear. The previous search box with the list of matches was MUCH nicer. Presentation should be about content and functionality! NB: as a visual impaired user, the Erlang docs stand out for their ease of use and readability. Well done!
  • OTP as a library is good enough as it is. The current set of modules covers most of my needs (80-90%) and I really hate having to look around for what dependency I should include now, which one is better, see if it is maintained, what quality the docs are etc. I just love Mnesia and digraph or Xmerl because they are just there when I need them.
  • All those BEAM languages are nice as an intellectual exercise for a few of us but are very unlikely to become mainstream or get enough momentum to be considered for serious work.
  • Erlang / OTP for me is an engineering tool, and as such it is the best thing around. Donā€™t go the Python way, which used to be nice until they started adding stuff somewhere around version 2.4.

So what can be improved?

  • Adopt rebar3 as the standard tooling. But donā€™t make it do to much magic! Magic is not engineering. An engineer wants predictability and simplicity
  • Update the Erlang textbooks, such as ā€œErlang programmingā€, ā€œDesigning for Scalability with Erlang/OTPā€, ā€œErlang and OTP in Actionā€, Joeā€™s and Ferdā€™s delightful books. Ferd, Tristan and a few others have created wonderful stuff which are really, really really good. Do not reinvent stuff, update it and keep it up to date. Maybe make these texts available for free since it seems to be a barrier for some people to pay a few bucks. Judging some of the recent questions posted, some people are just not willing to take the effort to ā€œlearnā€ stuff.
  • Put up a statue for the Erlang team andā€¦
  • Abandon Erlang Forums and keep using the good old mailing list. Email seems to be the perfect lists:filter/2 for clutter and noise and allows you to focus on your work for more than five minutes (sorry).

Have a nice day!

\Frans

9 Likes
  • something like eleveldb but actually built in, maybe even into dets or mnesia
  • remove the annoying GB limits in dets
  • native ability to build & ship a single binary, instead of just a release; Iā€™m aware of escripts, and other addons but a proper ā€œgive me a go or rust style binaryā€, statically include any NIFs and stick any BEAM files on the end as an .ez
7 Likes

Can the official publish a book like this: https://github.com/happi/theBeamBook怂
Very helpful for deep learningļ¼ļ¼ļ¼

5 Likes

More than ā€œmore of moreā€, my personal peeve with OTP libraries is that it denies me the ability to use pattern-matched selective ā€˜receiveā€™, just like raw Erlang code was supposed to. The handle_{ call | cast | info } event callback functions are so-o-o 1990ā€™s! MS-Access BASIC anyone?! Ugh!

I do have a prototype behaviour that restores selective receive to my OTP-supervised code, and I enjoy creating state machines with it. Maybe someone here might be interested to kick the tyres, and tell me how to improve it?

3 Likes

eGbh I rewrote otp behaviourļ¼ŒOnly minor changes.

3 Likes

People have tried but it is a hazzle. I know Klarna tried for some time but ended up implementing new backend for mnesia like elevldb.

But I would like to see some love to mnesia. I think it is really great that it is included and it works very well.

9 Likes

Definitely not what I meant, thereā€™s never been what Iā€™ve seen, experienced, nor what history shows. Rather, I merely took an optimistic view that more optimizations surely must be possible.

Every time I think something has been pushed as far as it get can be pushed (and not just in regards to Erlang) Iā€™m proven wrong :slight_smile:

4 Likes

Iā€™d love to see better HTTP clients.

Thereā€™s no good way to use OS provided cacerts, and most of the clients are insecure by default and require a lot of knowledge to secure. Hackney is an exception but itā€™s HTTP1 only, Iā€™d love the same for HTTP2 and websockets.

It has been really cool to see Elixir making big improvements in this area (and in terms of their performance), but I am a bit sad that these libraries canā€™t easily be used from Erlang, Gleam, etc.
If I had the time Iā€™d like to port one over to Erlang or Gleam so we can all use it.

10 Likes

Before reading my wish, understand that I really know how far fetched this is based on how the scheduler works, but I have spent 3 years refining what I need for a project and this is my magic wand want.

I would actually want the ability to run some scripting language as part of a process. Imagine I can spawn a process that has some micropython interpreter that will allow me to run small scripts that users can create themselves. The project Iā€™m busy with requires thousands of ant-like erlang programs to communicate and collaborate and erlang is PERFECT for this massively concurrent message based co-operative stuff but to make it accessible for users that want to experiment with small plugins that make each individual ant uniquely intelligent, they need to add this intelligence as C, python, or LUA.

I have a good understanding of how BEAM works and why this isnā€™t possible considering the scheduler and other stuff. But I refuse to use any other language for the foundation of this program, there is just nothing like it out there.

I know I can run a lua/python/JS interpreter and allow each ant to pipe a small bit of code it wants the result of, but if one person writes bad code it will crash/block all other ant plugins. I want an ant to die, or restart its interpreter if the plugin is bad, not bring down the whole nest.

3 Likes

Have you seen Luerl? It sounds like it may be what youā€™re after.

3 Likes

Yes, currently I have to run a NodeJS server to use the stuff I want like socket.io then TCP it to erlang.

4 Likes

I actually looked at it! I know I had some good idea of why I didnā€™t start adopting it but I will have to go back to my notes to give you a good answer. I am considering putting one of my interns on this to write some tests on the efficiency/usability.

4 Likes

Have you tried pyrlang? GitHub - Pyrlang/Pyrlang: Erlang node implemented in Python 3.5+ (Asyncio-based)

3 Likes