OTB decision on four proposed extensions to the lists module

Erlang News & Announcements Erlang Notices
1

In today’s meeting of the OTP Technical Board we discussed and decided on four proposed extensions to the lists module. Two were rejected and two were approved (with suggested modifications).

lists:transpose/1 – rejected

This pull request was discussed in a previous OTB, but we didn’t reach a final decision. After some discussion and after considering the feedback in the use case thread, which reached the conclusion that the it would not be used frequently enough to warrant inclusion in the lists module.

lists:foldwhile/3 – rejected

We rejected it for the following reasons:

  • Using foldwhile does not save much code compared to writing your own tail recursive function and the resulting code could become less clear.

  • The convention of returning a tuple with cont or halt in the first element is new convention for the lists module.

  • We understand why Elixir has a similar function (Enum.reduce_while/3) because it is convenient to use in a pipeline. Since Erlang does not have a pipe operator, that usefulness does not transfer to Erlang.

lists:uniq/1 – approved with modifications

Approved but with the following changes:

  • The implementation should be map-based for efficiency. See the uniq_one_by_one/1 function in the benchmarking thread for lists:uniq/1.

  • Another function uniq(Fun, List) should be added.

  • It should be documented that both uniq/1 and uniq/2 will keep the first occurrence of an element when there are duplicates.

We also discussed the name (uniq vs unique) and decided that it would be better to have the same name as in Elixir (uniq). There is also a Unix command called uniq (although they don’t do exactly the same thing).

lists:groupby – accepted with modifications

We discussed whether the best place for the function would be the lists module or the maps module. We agreed that it would fit into either module, but in the end we decided that putting it in maps had some advantages:

  • While the return values for the lists module are not entirely consistent, they tend to return either one or more lists (wrapped in a tuple), or an element from the list. Therefore, adding a function that returns a map introduces as slight inconsistency.

  • By placing it in the maps module, it would be possible to place similar functions in modules such as gb_trees or array. Each module would return its own data type.

  • The lists module is already quite large with many functions, while maps have fewer functions.

The sole disadvantage that we could find for not putting the function in the lists module is that most people would expect to find it in the lists module because it takes a list as input.

If the name would be maps:groupby one could reasonable think that it would take a map argument. Therefore, we think that a better name that makes it clearer what it does is maps:groups_from_list.

17 Likes
2

Thank you for taking the time to explain the OTBs rationale as well as the decision.

3 Likes
3

Thank you for clear and concise answers!
I wonder if there was any discussion on maps:inverse function that swaps keys with value (resolving duplicates with callback). Something like this one:

inverse(Map, Fun) when is_map(Map), is_function(Fun, ) ->
    maps:fold(
        fun(K, V, Acc) ->
            maps:update_with(V, Fun, K, Acc)
        end, #{}, Map).
3 Likes
4

No, we didn’t discuss maps:inverse.

2 Likes
5

@max-au @bjorng is/was there a PR for such an addition (maps:inverse)? I couldn’t find any :sweat_smile:

3 Likes
6

Not as far as I know. I assumed that @max-au referred to some suggestion posted in the discussion thread for some other pull request or possibly in one of threads on this forum.

2 Likes
7

@max-au I have an production used implementation of inverse/1 here mapz/mapz.erl at master · eproxus/mapz · GitHub

But I like your implementation more, it’s more flexible!

3 Likes
8

In fact, a better implementation might be something more like:

inverse(Map) -> inverse(Map, fun(Old, _New) -> Old end).

inverse(Map, Fun) when is_map(Map), is_function(Fun, 2) ->
    maps:fold(
        fun(K1, V, Acc) ->
            maps:update_with(V, fun(K0) -> Fun(K0, K1) end, K1, Acc)
        end,
        #{},
        Map
    ).

This way one can decide to keep either the old or new key.

2 Likes
9

I see :slight_smile:

But that gives raise to an interesting question in regards to naming and naming consistency again, maybe worth discussing at OTP/OTB in view of future additions: Should the name be inverse or invert here, ie should a function name align with what the function delivers (the inverse map of the given map), or describe what it does (invert the given map)?

In OTP, we have both, eg maps:keys or maps:values aligning with what the functions deliver, as well as maps:fold or maps:filter which describe what the functions do.

It is probably not feasible to adhere strictly to either one form or the other, more so since we have a mix of both all over the place already. But it may be good to formulate a preference if there is a choice with both types of names equally sensible.

4 Likes
10

@AstonJ Do you think it would be useful to have an area of the forums that collates reference implementations of common, concise and elegant example solutions like the above? I’m thinking of something like a curated list of gists.

The purpose would be to both provide a learning resource and a de-facto way of implementing a common pattern in user code, without the tighter constraints required for it to be in the standard library.

4 Likes
11

Naming is hard!

2 Likes
12

Yeah :sweat_smile: Good thing I have no children, they would probably be called “You Over There” and “The Other One” :grimacing:

3 Likes
13

That would be a nice thing to have, but require a number of dedicated curators (depending on how much engagement the area receives), ie people to decide what is good and bad (on what grounds exactly?), and to see to good naming (again) for the threads so people can find what they’re looking for and at the same time prevent double posting. Otherwise, it might quickly turn into something like this: BOFH: We want you to know you have our full support • The Register :roll_eyes: (Yeah, I’m exaggerating :wink: )

2 Likes
14

Oh, so you’ve met my 2 children, and my dog? 3 names seemed excessive when time division multiplexing exists.

2 Likes
15

I often have a need to keep the list of keys when they have duplicate values. E.g. when I am inverting a dependency tree stored as a map #{two => one, three => one} into #{one => [two, three]}.

For that case my conflict resolution Fun is similar to store(New, Old) -> [New | Old].

Somehow I am not able to find previous discussion on the topic, but I clearly remember it happening somewhere. @bjorng should I come up with PR for OTB to start discussions, or we can have the OTB decision first to avoid unnecessary coding if this is not going to be accepted?

1 Like
16

Just created a new category Phil :023:

This can house things like how-to guides or any other kind of tips or snippets people want to share. It can be used alongside our ‘glossary’ type section:

Where rather than posting those tips yourself, you can invite others to submit comments, eg: How would you achieve ____ in Erlang?

:003:

2 Likes
17

An complete implementation is not necessary for OTB to discuss an addition to the standard libaries. What is more important is a good motivation for why the new function is useful and example of use cases and perhaps a motivation for the name of the function if several reasonable names are possible. A thread in this forum could be a good place to discuss and suggest new functions and gather examples of use cases.

(Regarding maps:inverse, there is a similar function in sofs called converse/1. Either the same name should be used for consistency or there should be a motivation for why the names should be different.)

2 Likes
18

I was not aware of sofs:converse. The reason to name it inverse is, other languages call it that way (e.g. Java’s Map). Also, it feels in line with inverting the 2D matrix (swapping rows with columns).

There are plenty of cases where inverse would be helpful. Classic example is turning dependency tree from a map “B depends on A, C depends on A” into a map of “when A is done, we can start B and C”. That is, #{b => a, c => a} turned into #{a => [b, c]}.

I also remember that a recent Dialyzer PR #5498 uses sofs:converse to achieve the same result, and I actually think that maps would yield higher performance compared to sofs.

3 Likes
19

inverse is (IMO) more intuitive than converse, which has other connotations in natural language.

The standard library functions are also quite inconsistent regarding whether they are verbs or nouns. On a quick scan of the maps module, the majority seem to be verbs and, arguably, some of the nouns are simply contractions of get_xxx (e.g. maps:values). This actually bothers me less than I feel it ought; perhaps as @Maria-12648430 mentioned, consideration has historically been given to what the function delivers vs what it does. Or, like lists:reverse, it works both ways.

3 Likes