What Erlang/BEAM related stuff are you doing?

This week I’ve been experimenting with using Google Sheets as a data store for a small Gleam web app.

Querying is obviously very limited, but as a zero cost place to store a few thousand rows of data it seems pretty good so far. And it comes with a built in admin and reporting interface! The oauth2 dance to get authenticated is a hassle though.

I was originally inspired by some people using Airtable to do the same, but I was put off by the very low API rate limit.

2 Likes

You might find this from Joe Armstrong interesting @lpil

Databases are basically rectangular tables of cells, where the cells contain very simple types like strings and integers - every time you access a row of an external database this list of cells has to be converted to beam internal data structures - this conversion is extremely expensive.

The best way to persist data is in a process - then no conversion is needed, but this is not fault tolerant - so you need to keep a trail of updates to the data and store this on disk.

Often you don’t need a database for example you might like to have a system where you store all the user data as in the file system with “one file per user” this will scale very nicely - just move the files to a new machine if you need more capacity.

Erlang has two primitives term_to_binary and the inverse binary_to_term that serialise any term and reconstruct it - so storing complex terms on disk is really easy.

I have mixed feelings about databases, they are great for aggregate operations (for example, find all users that have these attributes) but terrible for operations on individual users (where a single file per user is far better).

If I were designing a new system I’d go for ‘one file per user’ as much as possible and try to limit databases for operations over all users.

From: https://elixirforum.com/t/acceptance-of-erlangs-term-to-binary-and-vice-versa-in-elixir/11034

6 Likes

Building a browser based implementation of the now defunct Battlemail Kung-Fu game

Where can i buy that cup?

3 Likes

I let it print :wink:
In my old company, each of our team made another cup, with logos from tech that got forbidden by the management. On my cup is RabbitMQ and Erlang, on another cup was kube and gRPC…

6 Likes

Writing a library for pretty printing 32bits and 16bits floats to string for Nx at GitHub - DianaOlympos/ken: Float to string conversion in erlang for ff64, 32 and f16 using Ryu, which is going… well not that well. I have been at it for months and the tests are still kicking my ass due to the pain in the ass of well defining the problem well.

Other than that, i am slowly bringing GitHub - LivewareProblems/hyper: Erlang implementation of HyperLogLog this into shape for the 20s, but i still need to do some work on it.

And i am still trying to think of the best way to cross compile erlang and embed it in a release in the Nixpkgs ecosystem because i want to be able to produce release artefact for both Graviton2 and x86 for AWS. No good solution yet.

Help and PR appreciated ofc

3 Likes

That’s too bad, I would of like to buy that cup to drink my soda while coding. :wink:

3 Likes

What Erlang/BEAM related stuff are you doing?

Reading Erlang syntax. Even if Elixir is enough in its own, maybe one day i may be stuck with an Elixir dependency that has sprinkles of Erl syntax. Wouldn’t hurt to learn it sooner than later.

Also, do we have Erlang version of Dave Thomas’ Exexif Elixir library for extracting exif data from an image?

2 Likes

You’re being modest. KRED sources are up to 1.8MLOC now, so I usually round up to 2MLOC :slight_smile:

3 Likes

Check erlang_exif | Hex

We are using it in production.

3 Likes

Just finished implementing my 404 page.

2 Likes

The main Erlang/OTP project I’m involved with would be ejabberd, but lately I got sidetracked to work on a TURN server called eturnal. I like how the actual protocol code is less than 2k lines of straightforward Erlang and yet offers everything modern A/V clients need. It’s being used in production by various sites and seems to scale well. I’m actually pondering with supporting proper release upgrades given that those would be especially useful for TURN, where service restarts kill the running calls.

6 Likes

This is most of what I’m doing, these days.

Public - FOSS

Internal - company (@Miniclip)

  • previously, creating/maintaining several game backends (distributed Erlang) + backoffices (HTTP API using cowboy)
  • maintaining a consents’ manager + privacy framework (HTTP API using cowboy)
  • maintaining a (legacy) mass push notifications’ system
  • creating/maintaining the main CI-specific libraries used by all others
  • previously, creating/maintaining over 80 libraries that support the previous (e.g. Datadog client, JWT wrapper, Redis wrapper, Facebook API wrapper, Google API wrapper, Apple API wrapper, AWS+erlcloud stateful wrapper, pub/sub, …)
5 Likes

right now i’m working on a library to do something similar to what pathom does in clojure. it started as an elixir project, but since i didn’t use anything particular to elixir, i rewrote it in erlang. i’m just finishing documenting it. :thinking:

2 Likes

Start studying BEAM / Erlang because I want to build a similar WhatsApp issue.

2 Likes

In the commercial realm, occasional freelance work, as long as it is interesting. (And as @juhlig said elsewhere, no company that values its pride should think of hiring me :grin:)

In the realm of FOSS, I mostly like to contribute to OTP and whatever other (Erlang-related) project sparks my interest. Generally, I like to find ways to poke holes in shiny things :innocent: If I love them enough (like Erlang :smiling_face_with_three_hearts: ), I also help in closing them up.

That, in a nutshell, is what I do :wink:

10 Likes

Tested a lot of packages which were entered for the SpawnFest 2022 contest as a judge. It’s a really good opportunity for me to learn what’s up in the world or Erlang, Elixir, and BEAM.

4 Likes

Just finished reading Gene Sher’s book “Handbook of Neuroevolution Through Erlang”. Good book, very interesting. Wish the code base was updated as it is a decade old! But still would recommend the book for those interested in using Erlang for ML. It definitely has some advantages.

1 Like

I have worked with distributed systems and familiarized myself with various problems that occur in distributed systems. Attempting to solve them using Erlang.

Trying to use more what otp has to offer by using other behaviours such as gen_statem and gen_event which are not used so often.

1 Like

We’re building an Ethereum Virtual Machine indexer using Erlang with a CQRS architecture. Devs can subscribe to events for a particular smart contract (like an ERC-20’s transfer events) and keep track of state of things like wallet balances. Using the distributed capabilities built into Erlang, the dev can have their entire app model running locally rather than on the remote server so they can hit the service as hard as they want and it won’t effect other users.

2 Likes

Wondering how to improve the performance of the scenario: Why is there no ordered_bag type ETS table?. Thinking whether it is feasible is add update_spec to ETS, similar to match_spec but for in place updating.

1 Like