Is it possible to add piping ala |> to Erlang?

Chat / Discussions
1

I am curious. Is it possible to add piping to Erlang with a |> operator?

I work in Elixir but am considering doing something in Erlang, and I know that the number one thing I’ll miss in Erlang coming from Elixir is the |>, especially since I’ve spent some time in F# as well.

I’m not familiar of the details of how Elixir does it other than the operational notion that the output from a pipe is inserted as the first argument in the following function. I’d be curious if it’s possible in Erlang and if there’s been any thoughts about adding it.

7 Likes
2

Technically it should be possible, but in reality it won’t really be usable since Erlang lacks the argument order consistency that Elixir brought when it was built.

The pipe operator relies specifically on the first argument being the structure being operated on. Unfortunately this is not always the case with Erlang functions.

Leonard

3 Likes
3

You can get the general idea of a pipe with a fold:

pipe(Data, Funs) -> lists:foldl(fun(F, Arg) -> F(Arg) end, Data, Funs).

%% And then usage like this:
pipe(0, [fun(X) -> X + 1 end,
         fun(X) -> X + 2 end,
         fun io_lib:print/1]).

But as @LeonardB points out, you might have to do a lot of reordering of arguments.

3 Likes
4

I remember Joe Armstrong was saying in Erlang it’d probably way to require the argument position to be explicit using &

Data
|> lists:map(fun mapper/1, &)
|> save(&, Database).
8 Likes
5

Thanks for the discussion everyone.

From what I can see, there are three possible solution layers:

  1. Simply add a pipe operator like |> to Erlang with no change in argument order for existing functions. The operator could default to insert the previous output into either the current first argument or last argument. This would allow new development to form around whatever standard is picked.
  2. Do (1) but then introduce a “pipe insert” operator, like the already suggested &, that allows inserting the previous output into whatever argument slot wanted. It could either be required to be provided or a default of last or first argument would be invoked in the absence of the insert operator.
  3. Introduce the pipe operator like in (1) but then change all Erlang core functions to follow whatever argument insert convention. This is obviously quite drastic and likely to be fraught with danger.

I really like (2) and what I was thinking of once it was mentioned that Erlang doesn’t have any strong convention of argument positions (I’m just now picking Erlang back up but am familiar with Elixir). In fact, the pipe insert operator really should exist in other languages that already have the pipe operator, like F# and Elixir, as one still comes across times where it would be nice to have.

Is there interest from the Erlang core team about this idea? Has it been hashed out before?

(2) seems like a nice solution in that it would greatly improve code going forward but wouldn’t cause any issues with existing code.

6 Likes
6

The problem with (1) is that the order is not consistent. There are modules where most functions have their “subject” in the first argument (especially if it’s a process-like thing e.g. gen_server but also ets), there are some that have it only in the last argument (like queue), and there are some that have additional forms of functions of the same name that append the additional parameters (e.g. maps with get/2 and get/3).

Regarding (2), this is actually (in my opinion) the more generic subject of partial function application which I’d really like to see in Erlang. The expression

lists:map(fun mapper/1, &)

would be equivalent to

fun (X) -> lists:map(fun mapper/1, X) end

and you would define the pipe-operator explicitly as

V |> F := F(V).

This wouldn’t even need Elixir’s magic of implicitly using the first parameter, which I would like as Erlang has been nicely explicit so far :slight_smile:

3 is out of the question IMO.

4 Likes
7

There is this experimental parse-transform that does more-or-less what is discussed here. Never tried it though, but looks interesting GitHub - fenollp/fancyflow: Experimental library to bring pipe and maybe operator equivalents in Erlang

4 Likes
8

This thread might be off interest too.

2 Likes
9

You can take a look at the etran project, which implements the erlpipe parse transform that is “almost” functionally equivalent to what |> operator does in Elixir. E.g.:

test1(Arg1, Arg2, Arg3) ->
  [Arg1, Arg2]                                   %% Arguments must be enclosed in `[...]`
  / fun1                                         %% In function calls parenthesis are optional
  / mod:fun2
  / fun3()
  / fun4(Arg3, _)                                %% '_' is the placeholder for the return value of a previous call
  / fun ff/1                                     %% Inplace function references are supported
  / fun erlang:length/1                          %% Inplace Mod:Fun/Arity function references are supported
  / fun(I) -> I end                              %% This lambda will be evaluated as: (fun(I) -> I end)(_)
  / io_lib:format("~p\n", [_])
  / fun6([1,2,3], _, other_param)
  / fun7.

test2() ->
  % Result = Argument   / Function
  3        = abc        / atom_to_list / length, %% Atoms    can be passed to '/' as is
  3        = "abc"      / length,                %% Strings  can be passed to '/' as is
  "abc"    = <<"abc">>  / binary_to_list,        %% Binaries can be passed to '/' as is

  "1,2,3"  = {$1,$2,$3} / tuple_to_list          %% Tuples   can be passed to '/' as is
                        / [[I] || I <- _]        %% The '_' placeholder is replaced by the return of tuple_to_list/1
                        / string:join(","),      %% Here a call to string:join/2 is made

  "1"      = [min(1,2)] / integer_to_list,       %% Function calls, integer and float value
  "1"      = [1]        / integer_to_list,       %% arguments must be enclosed in a list.
  "1.0"    = [1.0]      / float_to_list([{decimals,1}]),
  "abc\n"  = "abc"      / (_ ++ "\n"),           %% Can use operators on the right hand side
  2.0      = 4.0        / max(1.0, 2.0),         %% Expressions with lhs floats are unmodified
  2        = 4          / max(1, 2).             %% Expressions with lhs integers are unmodified
5 Likes
10

We were discussing this by chance today at work. We ended up at something along the lines of your option 3. Other languages have multiple standard libraries in common use. Ocaml seems to generate a new stdlib for pretty much every major user.

If there was an alternative stdlib that had consistent argument order, and the language supported the pipe operator I expect new projects would tend to use the new stdlib

Of course this would require the magic faery to sprinkle the resources and motivation to make it happen. But fun to imagine.

2 Likes
11
  1. Leave well alone. :sweat_smile:

The primary (and AFAIK, only) argument brought to the table in favor of the pipe operator is roughly that…

d() |> c() |> b() |> a()

… looks somewhat nicer than…

a( b( c( d() ) ) )

… does. Agreed, but I pose the question: how often have you seen anything like that in a real Erlang program, and how often have you yourself written anything like that? :smile:

More serious, it is a recipe for confusion.
If I look at some line of current Erlang code and see a call to a function with n arguments, I can be sure that what is called is that n-ary function.
If we had a pipe operator, I would always have to check if there wasn’t a pipe operator somewhere before that or not, to see if what is called is the n-ary or really the (n+1)-ary function. The equivalent of always having to look over your shoulder no matter where you are. :face_with_spiral_eyes:
The & placeholder suggestion might lift that to the degree that I at least know what function is actually called, but I’d still have to look around elsewhere for what goes in the place of that nondescript &, instead of having either a (hopefully aptly named) variable or function call right there, in place.

In a nutshell: I don’t see any compelling reason why Erlang needs a pipe operator, in any form. “Make Erlang look more like Elixir” doesn’t count as a reason :stuck_out_tongue_winking_eye:

14 Likes
12

Yes, indeed :smile:

1 Like
13

RS232C is all the standard we’ll ever need lol

2 Likes
14

Agreed, but I pose the question: how often have you seen anything like that in a real Erlang program , and how often have you yourself written anything like that ?

Piece of code I’m looking at today:

    State1 = #state{ top_block_hash     = TopBlockHash,
                     top_key_block_hash = TopKeyBlockHash,
                     top_height         = TopHeight,
                     consensus          = Consensus},
    State2 = set_option(autostart, Options, State1),
    State3 = set_option(strictly_follow_top, Options, State2),
    {ok, State4} = set_beneficiary(State3),
    {ok, State5} = set_sync_mode(State4),
    State6 = init_miner_instances(State5),
    State7 = set_stratum_mode(State6),

This is from the Aeternity codebase, but it’s a pretty common pattern that would be a lot cleaner with pipes. Renumbering the States to fit another step is annoying and for sure error prone. Alternative:

    #state{ top_block_hash     = TopBlockHash,
            top_key_block_hash = TopKeyBlockHash,
            top_height         = TopHeight,
            consensus          = Consensus}
    |> set_option(autostart, Options)
    |> set_option(strictly_follow_top, Options)
    |> set_beneficiary()
    |> set_sync_mode()
    |> init_miner_instances()
    |> set_stratum_mode(),
6 Likes
15

Take a look at the Datum library, specifically the category pattern.

2 Likes
16

I’ve seen it. It doesn’t feel very Industrial programmer friendly:

We are using a category pattern to implement the category of monadic functions, which generalises ordinary functions

For me the interesting questions about consequences of adding the simplest possible pipes in Erlang (Elixir style pipes probably come close) are:

  • How would Erlang programs be written differently if it had a pipe operator
  • Would those programs be better or worse
  • If “better” would that be enough to outweigh the additional load on the language

One downside might be that people end up writing all their code in an unnatural manner just to be able to use pipes (i.e. always use one big state threaded through rather than more focussed data structures that better represent their data)

Another could be around error handling

Upsides could end up being much more consistency throughout a large project of argument order and composability of all the little functions that make up a system, and arguably readability. The example I posted for me makes the code much cleaner. For example it’s obvious at a glance which of those composed functions use side effects (i.e. almost all of them!)

3 Likes
17

As far as I can tell, there’s only three primary ways to do staged function calls in a text-based language:

  1.  h(g(f(x)))

  2.  output_f = f(x)
 output_g = g(output_f)
 output_h = h(output_g)

  3.  f(x) |> g(&) |> h(&)
    or better yet
    f(x)
|> g(&)
|> h(&)

(where I’m using & as a pipe output placement operator).

I haven’t programmed in Erlang much yet (maybe will soon!), but I can only assume that without a pipe operator (1) and (2) are used all over the place. I think the pipe operator is the most clear when there’s a clear flow of data and ordering, and it keeps one from having to create intermediate bindings or having nested function calls. It can in fact also help cleanly organize new code in that you can do “pipeline-driven development”.

In both F# and Elixir, it’s pretty easy and actually quite common to mix and match all three options shown above. The point is that it’s nice to have all three at your disposal and to not be forced into one or the other when another is the better choice.

A nice example, I think, is an assembler I wrote in F# at one point:

let assembleFile (filePath: string) =
    let directory = Path.GetDirectoryName(filePath)
    let filename = Path.GetFileNameWithoutExtension(filePath)
    let hackFilePath = Path.Combine(directory, filename + ".hack")
    filePath
    |> readLines
    |> parseLines
    |> buildSymbolTable
    |> translate
    |> writeAssemblyFile hackFilePath

There, (2) and (3) are mixed nicely.

The & placeholder suggestion might lift that to the degree that I at least know what function is actually called, but I’d still have to look around elsewhere for what goes in the place of that nondescript &, instead of having either a (hopefully aptly named) variable or function call right there, in place.

Do you really need to do that all the time when functions are named appropriately? The existence of a pipe operator in a language does not prevent you from creating a named binding for a function where the output is not clear.

“Make Erlang look more like Elixir” doesn’t count as a reason :stuck_out_tongue_winking_eye:

That’s absolutely and definitively not the reason.

The reason is that pipe operators are useful as discussed above. F#, Elixir, Racket, Clojure, PowerShell etc. have them because they’re useful, not because of fashion. It’s just like pattern matching, once you use pattern matching or piping in a language, it’s hard to move to one that doesn’t have them.

It seems to me that it could be added to Erlang with little fanfare.

5 Likes
18

Having written F# and then Elixir, I don’t experience this. It always goes in the same spot and you’re not running around searching for pipes.

I’ve even started using Magritte, which allows piping into a chosen position, which is what Erlang would need and I find it very nice to read. Very explicit.

There are people who like to pipe everything, but I don’t see it in the libraries I poke around in.

4 Likes
19

My experience as a long time (ehem) Erlang programmer who started to use Elixir for a new SaaS project was that there was a short period of “looking over shoulder”, but quite soon I had sufficient familiarity with the idiom that I also didn’t think of it as odd any longer, but instead as a really useful tool.

This step of “an extra thing to become familiar with and mentally embed” is part of what I was referring to as the added load on the language.

But at this point it’s pretty much the only thing I miss from Elixir now I’m back to Erlang as the day job.

My personal take is that it would be a great addition to Erlang with low extra cognitive load and high utility.

4 Likes
20

I especially miss it in the shell. It’s super handy to build up some exploratory query / transformation without having to manually skip around adding parentheses. Just |> the_new_op(). and done.

4 Likes