What would you like to see in Erlang or the BEAM world?

Abdelghani · December 13, 2021, 10:27pm

surely 6

Hanabi · December 13, 2021, 11:00pm

official json support
pipes
what @Gwaeron mentioned about “missing” functions in the lists module. I’d also like to add transpose([list()]), window functions like sliding,group,group_with and probably something I’m forgetting rn
what @lpil said about using native ca-certificates
I know this is out of the question but a JIT that works on rv64gc and ppc64el would be nice too : /

sey · December 14, 2021, 9:08am

I love Erlang, coming from JVM’s Scala, my mind is blown away already however I think static typing is important. It doesn’t need to go into too much types theory (we have Haskell for those type of programmers). However it needs to support the basic types, and perhaps some basic self defined types; so that during the compilation some errors could be shown to the user, as well as people who just start learning the language, could use these types to reason about different features of the language.

Gwaeron · December 15, 2021, 2:54am

It evidently doesn’t need to support basic types or any other typing.

While the compiler doesn’t do what you’re asking for, you should have a look at the very useful static analysis tool dialyzer which is definitely up to the task!

fuelen · December 16, 2021, 4:40pm

Why? It is really easy to add JSON library as a dependency to the project.

erszcz · December 16, 2021, 5:55pm

Performance has not stopped Ruby or Python (TIOBE #1 as of Dec 2021) from mainstream popularity. And Erlang is more performant than both of those even without the JIT

starbelly · December 16, 2021, 8:10pm

Per the last time this was attempted and my thoughts :

github.com/erlang/otp

Add BIFs term_to_json and json_to_term

erlang:master ← dhull:master-json

opened 02:01AM - 15 Feb 19 UTC

dhull

+3531 -1

This branch implements two new BIFs, `erlang:term_to_json` and `erlang:json_to_t…erm`, for converting between Erlang terms and JSON-encoded binaries. These BIFs are well-behaved: long running calls periodically yield to other processes. The BIFs are fast: on my data, encoding is about five times faster and decoding is about three times faster than the other libraries. See the [json.org](https://github.com/dhull/otp/blob/master-json/json.org) file for more information.

Taure · December 16, 2021, 9:52pm

github.com

JanHenryNystrom/jhn_stdlib/blob/master/src/json.erl

%%==============================================================================
%% Copyright 2013-2021 Jan Henry Nystrom <JanHenryNystrom@gmail.com>
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
%% You may obtain a copy of the License at
%%
%% http://www.apache.org/licenses/LICENSE-2.0
%%
%% Unless required by applicable law or agreed to in writing, software
%% distributed under the License is distributed on an "AS IS" BASIS,
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%==============================================================================

%%%-------------------------------------------------------------------
%%% @doc
%%%  A JSON library based on:
%%%    The application/json Media Type for JavaScript Object Notation (JSON)

This file has been truncated. show original

This is the only json module I have used the past years. I like it because it is done in Erlang without NIF. It is fast, but not sure how it stands against NIF versions.

SisMaker · December 17, 2021, 2:04am

Different programming languages have different application scenarios, and Erlang and Python are not used in the same scenario most of the time. Of course, speed isn’t just a factor in choosing a programming language. But in general, the faster the better, especially since Erlang is used for concurrent distributed programming and often handles a lot of input.

elbrujohalcon · December 17, 2021, 8:06am

A few years ago, I wished for a couple of things that we already have (like a formatter, a dead-code checker, even a spellchecker, FWIW - thanks, @vkatsuba)… But there is one thing that I’ve been wishing for… for a as long as I can remember:

Better Dialyzer Warning Messages

I wish for…

Clearer descriptions
A way to figure out the context of a warning so I don’t have to deduct it myself through code investigation.
Basically, the actual implementation of otp/dialyzer_explanation.erl at master · erlang/otp · GitHub

SisMaker · December 17, 2021, 9:47am

i add something
1 can add one bif fun to return the term type
2 We used term_to_string/1 string_to_term/1 a lot in previous programming. Can they be implemented in bIF?
Term_to_string /1 is usually implemented using io_lib:format, but io_lib:format large term consumes a lot of memory. I optimized the version of this function, and after optimization, the memory consumption is 1/10 of the original, It would be even better if it were implemented with a BIF.
3 Can achieve [xx | | One < - Lists] for map version, * Traversing a map efficiently and simply does not seem to be an easy task
4 I recently learned to use Redis and found it to be really simple, easy to use and efficient, how wonderful it would be if Beam integrated redis data structures into the Beam VIRTUAL machine

bjorng · December 17, 2021, 1:15pm

We have started to do some refactoring and cleaing up of the Dialyzer code for OTP 25. We intend to improve the messages, but it will probably not be in OTP 25.

Saprrow · December 19, 2021, 5:45am

I would like to be able to manipulate nested maps more easily. This would make a massive difference for data stores in ETS or gen_server states and allow my maps to be 1 to 1 mapping ‘haha’ with the JS objects in my interfaces.

Saprrow · December 19, 2021, 5:52am

I’m going to have a look! But if I’m correct, this makes the python program act like an erlang node? In which case any python scripts will need to be sent to this node to be run.

vkatsuba · December 19, 2021, 3:09pm

Erlang containerization implementation as Docket and Kubernetes

OvermindDL1 · January 3, 2022, 4:39pm

This is mine. The lack of good static types is the primary source for easily 90%+ of my runtime bugs… It is the compilers responsibility to check this, not a squishy human brainmeat!

Plus refactoring or maintaining code without static types is its own layer of hell… >.>

Plus with static types the JIT could do even better by using that information. Sure it might need a callback shim on a module being loaded in the system to do the appropriate conversions if types changed in versions, but that’s all doable as well and the JIT can always fallback to dynamic testing then based on the exposed types of both modules instead of needing to test the type of the data itself (removing that would be nice!). This would be a much much larger change for the language though to mandate static typing.

starbelly:

A slimmed down and more ergonomic standard lib and better UX in general. I think there’s a lot of simple things that can be done to improve the Erlang standard library. I think there’s a lot to learn from Elixir and other languages in general in this regard. One idea I had was an enum module. Of course, we wouldn’t have protocols and such, but simply a module that lets you interface with enumerables might not be a bad thing to have. There’s also lots of little enhancements we could make to existing modules IMO. I also fancy the idea of building on top of OTP to make a framework that nudges people down a “proper” OTP design path. I’m not terribly sure what that looks like without a lot of thinking, but I do like the idea. I used to not like these ideas as I thought the barrier to entry being high was a good thing, and I do think there’s some truth to that, but I think in retrospect it’s folly, it doesn’t solve any problems, and perhaps can even be seen as gatekeeping. I think there’s a lot more to unpack, but that’ll do for now.

For an enum module it would be fairly simple to do it for any type if the type included the module callback name (like via a record type, have the first atom be the name of the module and the name of the record) or potentially just pass it in to the calls as a witness. That would be a very nice general API and a set of user-extensible iterable types!

You still can selectively receive inside a callback inside a, for example, a gen_server just fine, that’s how most callback functions work (that’s even how calling the public functions on a gen_server work). Just you must handle all of them eventually and allow it to return system information as well.

+1

Not in the standard library however, this is something where there an be many different styles of handling for different performance considerations and more. Adding one to the standard library is how you get the horrible things like python’s json module and so forth. Keep these as libraries, sure you can have some officially supported ones, but it still should not be in the standard library.

Eh, a positive typer like dialyzer is not a replacement for static typing, it’s really bad at figuring out a lot of things (that have indeed caused me bugs in the past).

wojtekmach · January 4, 2022, 10:25pm

this + near instant boot time (say, <100ms) would be really fantastic for shipping CLI utilities. I know boot time doesn’t matter much for servers which the VM is optimized for but, you know, one can dream!

marcellanz · January 5, 2022, 9:34am

doesn’t matter much for servers which the VM is optimized for but, you know, one can dream!

Boot-times matter in other context in the cloud. I’d love to see “fast” boot of the VM as a target of interest.

garazdawi · January 5, 2022, 10:44am

On my machine (which is not a very fast Linux machine) it takes about 84ms to boot and shutdown an Erlang VM.

> time ~/erlang/24.1/bin/erl -mode minimal -noinput -boot no_dot_erlang -eval "erlang:halt()"

real	0m0,084s
user	0m0,107s
sys	    0m0,016s

Booting an escript archive is about 20ms slower, so around 105ms. Unfortunately, it is a lot slower on macOS at about 200ms, which I think mostly is due to DualMapping allocation adds 100 ms to JIT startup time on Mac OS · Issue #4911 · erlang/otp · GitHub. I didn’t test running the non-jit VM on macOS, but I suspect that it will be about as fast as Linux.

Of course, the more applications you start, the longer start time you get. A lot of the start time is spent doing code loading, so if we could find some way to preload/cache code loading we could probably cut that down. That is however not easily done as the loaded code today contains a lot of pointers to dynamically allocated memory (for instance literals) that would need to be handled somehow.

wojtekmach · January 5, 2022, 12:34pm

Thank you, this is good to know. On my M1 and OTP master (73737c1e3e) I see the following numbers:

with JIT:

$ time erl -mode minimal -noinput -boot no_dot_erlang -eval "erlang:halt()"
0,11s user 0,12s system 80% cpu 0,284 total

when built without JIT:

$ time erl -mode minimal -noinput -boot no_dot_erlang -eval "erlang:halt()"
0,08s user 0,12s system 129% cpu 0,156 total