Clarification on using records vs maps for state in modern Erlang

Questions / Help
,
1

Hi,

I’ve read Joe Armstrong’s opinion that “records are dead, long live maps,” and I understand the benefits of maps in terms of flexibility and introspection. (https://joearms.github.io/#2014-02-01%20Big%20changes%20to%20Erlang)

However, I’m specifically interested in using maps with predefined keys and types, like this:

type state() :: #{queue := queue:queue(), priority_map := #{...},...}.

When I look at recent Erlang codebases, like WhatsApp’s edb repository, I see both approaches being used:

This makes me wonder:

  • is there a community consensus or best practice today regarding when to use records vs maps for gen_server state?

  • Are records still preferred in some contexts (e.g., performance, pattern matching), or is it just legacy code?

  • Is using maps with strict typing (via -type) considered idiomatic now?

Any insights or recommendations would be greatly appreciated!

1 Like
2

Both have their place. Nuff said.

3

thanks for your reply

But if the structure is fixed and known ahead of time,I would have expected record to be the better fit, because they are checked at compile time.

So I’m genuinely curious: in such cases, why do some modern codebases choose typed maps over records? Is it about readability, tooling, or something else?

Would love to hear your take.

1 Like
4

Records are tuples with extra metainfo, and the metainfo is missing in many places. For example, in observer, you can see the state of a process, but usually this is what you will see:

{state,<0.127.0>,[],#Ref<0.2329095937.860225544.161198>,[<0.127.0>]}

{state,inet_db,inet_cache,inet_hosts_byname,inet_hosts_byaddr,
       inet_hosts_file_byname,inet_hosts_file_byaddr,inet_sockets,
       #Ref<0.2329095937.860094473.154504>}

{state,{conf,true,true},
       #{},[],[],[],nonode@nohost,<0.61.0>,<0.62.0>,no_trace,false}

You won’t know what they mean until you read the source code, of the right version.

It would be much easier if they use maps.

5

Read the memory usage section of the manual.
Records hold the values. They do not store the keys. There is one word of overhead for the tag.
Frames hold the values plus one word of overhead to point to a shared tuple of keys.
Maps hold the keys as well as the values and the keys are not shared.
So maps take about twice as much storage spCe as records.

With records, the offset of each field is known to the compiler so that accessing one field out of N is O(1) with a small constant factor.
With frames the cost of accessing K out of N fields was O(min(K+N,K.log N)).
With maps the cost of accessing one field out of N is constant expected time with a rather larger constant factor but hash tables being what they are, the worst case is O(N).

For debugging, the fact that maps are self-labelling and records are not pales into insignificanes compared tod withe the fact that you want custom printing anyway. This is the Prolog lesson and is why Prolog has the portray/1 command . It was observed in the early years of DEC-10 Prolog that printing a single call could give you a couple of screens of text. Data structures have only got bigger since then.

So overall records are smaller and faster. I’d say use map when you have to, as when the set of keys is not a smallish fixed set of atoms.

2 Likes
6

Please read the Map section of Efficiency Guide from Erlang document. The maps in Erlang may not be what you think it is. There are 2 kinds of representations inside beam, and keys are shared for small maps.

the performance of records compared to using small maps instead of records is expected to be similar.

For small maps (element num < 32), the overhead is tiny, the Big O should not be used on this situation.

For big maps, it’s a tree, you can update only part of the data instead of copying the whole data, which is what record have to do.

1 Like
7

And I would recommend you to read “the Erlang Rationale” by Robert Virding, there is an interesting history about record:

Records were added to solve a problem for our first customer. They were good users
and we really wanted to help them. What they wanted was:
•
•
Named fields in tuples, preferably with default values.
Not slower than doing it explicitly with element/setelement, if it was slower
they would not use it.
These requirements basically meant that everything had to be done at compile time,
finding fields at runtime would just not cut it. This forced records into being a
compile-time construction which went against the grain of rest of Erlang. Together
with macros they are still the only compile-time constructions in the language.
For simplicity it was decided not to create a new data type but to use tuples.

…

Some other possibilities for the design of record is also mentioned in that article.

8

Then the efficiency guide and the memory usage sections are contradictory and need correction.
I am aware of the rationale for maps; it’s not that different from the rationale for frames.
However, the context is strictly typed maps, and in that context the the quote from Robert
Virding doesn’t really apply.

1 Like
9

I still hold my opinion. For small number of fields, the runtime cost of maps is tiny. For big number of fields, the accessing speed of records is fast, but the updating cost of records is big.

We can not just think of records as struct in C, since record updating is memory reallocating plus copying. And states always need to update.

1 Like
10

To quote Brian Marick, “An Example Would be Good About Now”.
Updating multiple fields in a record can be done with only one copy,
but the main lesson is that large records are a bad idea anyway.
This also applies to maps used like records.

What we need are some benchmarks. I have to go and cook tea, and I’ve
got a lot of other stuff to
catch up on, but I’ll get back to it if I can.

Let us replace doxa with data whenever we can.
I would be delighted to be corrected by data here.

2 Likes
11

One case where records are always used is mnesia. A row in a table is (effectively) a record(), which by default has the same name as the table and fields key and val. If you really want to use map() you can embed it in val but you’ll always have a tuple()/record() representing the row:

1> mnesia:start().
ok
2> mnesia:create_table(foo, []).
{atomic,ok}
3> rd(foo, {key, val = #{}}).
foo
4> mnesia:transaction(fun() -> mnesia:write(#foo{key = 42, val = #{a => 1}}) end).
{atomic, ok}
5> mnesia:transaction(fun() -> mnesia:read(foo, 42) end).
{atomic, [#foo{key = 42, val = #{a => 1}}]}
2 Likes
12

This is the greatest answer I have ever seen !!!

“you won’t know what they mean until you read the source code, of the right version.“ –> I face this issue many times before and I have to waste my time to compare the record with its definition in source code.

2 Likes
13

Yeah, when I want to eliminate record from my head years ago, mnesia is the main thing that had stopped me.

14

15

Records are more compact, faster and protect against errors (typos in field names).

1 Like
16

One trick to avoid mistyping atoms (map keys in this discussion) is to define macros for them and always use the macros.

This works as the complier will complain if you mistype the macro name, as it will fail to find the macro definition.

The drawback is that you will have to define a bunch of macro constants like -define(key, key).

17

Well, if I do a simple redbug:start(“some_gen_server:handle_info”). I don’t think a custom format_status helps - but I may be wrong here.

18

Strangely I also came to this conclusion but in the context of sharing specific key names across multiple modules - i.e. when sharing data that is stored in a hash as a message (for example).

I extend the macros to include verbs, for example:

-define(GetPayload, <<"payload">> := Payload).
-define(PayloadIsSet, <<"payload">> := _Payload).
-define(SetPayload, <<"payload">> => Payload).
-define(AddPayload(V), <<"payload">> => V).

All of these are located in a header file and shared across all required modules.

Whether this is good or bad is relatively meaningless, its just an approach to avoid typos and be consistent in the naming and typing of keys in hashes (I started out with a mix of atoms and binary key names - before deciding on binary.)

19

We can dump the assembly code after JIT compiler to see why small maps are efficient.

Prepare

Let’s create a simple erlang module a.erl.

-module(a).
-export([map_test/1, record_test/1]).

map_test(Person) ->
    #{name := Name, age := Age, gender := Gender} = Person,
    {Name, Age, Gender}.

-record(person, {name, age, gender}).

record_test(Person) ->
    #person{name = Name, age = Age, gender = Gender} = Person,
    {Name, Age, Gender}.

Start Erlang with dumping enabled:

erl +JDdump true

We can call some functions in a to make sure it got loaded:

a:module_info().

Now there will be a new file a.asm generated. It’s 346 lines of assembly code in my x64 machine.

Let’s compare the data loading part of the maps and records.

Compare

The code for record (tuple) is super efficient, it’s just some load instructions.
OTP28 is smart enough to use SIMD instructions to make the code even shorter.

    vmovups xmm0, xmmword ptr [rsi+14]      # get_two_tuple_elements_sPSS
    vmovups xmmword ptr [rbx+8], xmm0

    mov r10, qword ptr [rsi+30]             # i_get_tuple_element_sPS
    mov qword ptr [rbx], r10

Now let’s see the code for maps. It’s some searches plus some load instructions.

L18:
    dec eax
    jl label_3				                # key not found
    cmp qword ptr [rsi+rax*8+6], 676939     # compare 1st key
    short jne L18
    mov rdx, qword ptr [rdi+rax*8+22]       # load    1st key
    mov qword ptr [rbx+8], rdx
L19:
    dec eax
    jl label_3				                # key not found
    cmp qword ptr [rsi+rax*8+6], 676875		# compare 2nd key
    short jne L19
    mov rdx, qword ptr [rdi+rax*8+22]		# load    2nd key
    mov qword ptr [rbx+16], rdx
L20:
    dec eax
    jl label_3                              # key not found
    cmp qword ptr [rsi+rax*8+6], 28619      # compare 3rd key
    short jne L20
    mov rdx, qword ptr [rdi+rax*8+22]       # load    3rd key
    mov qword ptr [rbx+24], rdx
    short jmp L16

There are jumps/loops (those jl instructions), but the loop variable eax is NOT reset after each key search!
keys to search are ordered so the search of all keys can be done in one iteration!

For our example, there are 32 cmps and 3 loads in the worst situation,
and 3 cmps and 3 loads in the best situation.

NO hash calculation, NO function invocation.

4 Likes