Is it possible to add piping ala |> to Erlang?

I love to see this pipe operator. It is going to be super useful and reduce few line of coding. In our case, many functions returns {ok,…} tuple as return values. It would be nice, param placeholder to be numbered (tagged) as &1, &2… etc… &1 is first value of output of previous fun and & may represents whole value. Just my 2cents.

2 Likes

When I implemented pipe support as a parse transform in etran, the lack of ability to use it in the shell was pretty regretful. I was a little short of time to create an EEP spec for the pipe feature, but maybe someone could take a lead and create one?

3 Likes

I’m trying to answer the previous posts all at once here (but not in order), so this might end up a bit confusing for readers… But here goes :sweat_smile:


Yes, and in that sense, this doesn’t bode well:

“If you have a pipe operator, everything looks like a pipe”. And while I’m only half serious here, I’m sure you can program sensibly with piping around too, it is wrong to blindly assume that everybody will use it sensibly. I guess it would warrant a whole chapter filled with DOs and DON'Ts (that, let’s be realistic, nobody ever reads anyway) and style guides, something that Erlang could avoid needing until today, by and large. And this…

… underlines my point further: Many functions return tuples, so in order to use them with pipes, we should provide a mechanism to rip values out of tuples. Again: for no other reason than to make pipes usable in yet another place.

One of the early suggestions to change the entire stdlib and pretty much everything to make it pipe-friendly (I guess/hope that wasn’t really a serious proposal) is another thing in the same line, it spells “everything must subvert to the pipe operator’s demands, it shall rule everything”.


What I really really like about Erlang is that there is not a myriad ways to do the same thing. There is one or two ways to do something, maybe three, one may be better than the other, but generally not much. You either do it this way or that way, or you don’t do it period.


The other thing I really really like about Erlang is that there is no magic going on that changes the semantics of what is written in one place because of something that is written in another place, close by or not. So if I see a call having three arguments in the list, what is called is a 3-ary function. But the pesence of the |> which is placed elsewhere changes this to 4-ary, with what is passed in there may come through a loooong pipe.


“Others have it” is not an argument if you ask me, I can name you a dozen other languages that don’t have them. Conversely, Java, Python, C++ etc have objects and whatnot and they are useful to the respective languages, but you wouldn’t suggest adding objects to Erlang I believe. Still, it would be the same argument as the one you made.

10 Likes

Ummm… @Maria-12648430 :flushed:
What’s wrong, calm down… It’s only a cute pipe operator :grin:

3 Likes

“It wouldn’t be hard to do → famous last words” (I don’t remember who said that to me and what the suggestion was I made at the time, but the phrase stuck) :grimacing:

3 Likes

Uh, yeah, sorry everyone for being edgy :cry: It’s a sore spot with me I guess :sweat:

4 Likes

There, that’s better *pat* :smile_cat:

3 Likes

@Maria-12648430 I could not agree more about this being an argument against it. People doing this sort of stuff, which obfuscates the structure and names of things so they can pipe everything is pointlessly confusing.

Where is the value in reducing the line count by one or two in order hide the structure of the data? Where!?!? IMO pipes are for transformation pipelines or for passing a single structure through various stages, not for making your function contents entirely a single pipedream.

8 Likes

I’m with @Maria-12648430. This in my book is an anti-pattern. You are losing the observability of each step and possibly creating a harder to maintain function structure through what is arguably laziness. Would I use a pipe operator if Erlang had it? Hell yes, I’m lazy too, and it absolutely makes for prettier code than the above. Does Erlang need it? I don’t believe Joe thought so, and his mantra was to make code beautiful.

Conversely, Java, Python, C++ etc have objects and whatnot…

Python has everything except the things I find most useful in a language with a C-like syntax. But best not get me started on my Python rant :wink:

We always call it out if someone in the team uses the word “just” in relation to any proposed task or change :wink:

9 Likes

@phild I would give you two :heart:s for your post if I could :blush: Was feeling pretty lonely with my opinion up till now :wink:

4 Likes

FWIW, I also don’t think we should add pipe to Erlang. When adding a new language feature, we need to ask ourselves how much “harmony” does it have with other features in the language. I don’t think piping would work well in Erlang due to the argument order, as mentioned by others. Whatever side you pick, pipe first or pipe last, it will cause friction with other parts of the language, and I think overall we’d be worse by adding it.

I also strongly dislike any syntax that says: pipe to this position. The point of piping is to make the argument and function call ordering readable. If I need to fully read the right side of |> to figure out where my argument goes, then I think we start losing the readability |> was supposed to give us.

I have the same concerns with additions such as two operators, one for piping first and another for piping last. Imagine something like:

Var
>>> pipe_last()
>> pipe_first()

I now need to think where my arguments go and the benefits of a pipeline are gone.

13 Likes

As much as my initial knee-jerk reaction is it would be nice to have, on reflection I believe it’d probably make code harder to follow.

Similar behavior can be achieved by folding over funs and provides, imho, more flexibility in the implementation since you can handle arguments/ordering whatever within your funs.

We have a few utility functions in our internal libraries specifically for this (which I suppose you could even call it a implementation of foldwhile being discussed in another thread.

Short example from a cowboy handler

-type state() :: #state{}.
-type creq() :: cowboy_req:req().
-type do_fun() :: fun((creq(), state()) -> {continue | stop, creq(), state()}).

init(CReq0, _Args) ->
    St0 = #state{},
    Flow = [fun allowed_method/2,
            fun check_apikey/2,
            fun check_authorized/2,
            fun check_limiter/2,
            fun read_params_or_body/2,
            fun handle_request/2],
    do_flow(Flow, CReq0, St0).

-spec do_flow(list(do_fun()), creq(), state()) -> {ok, creq(), state()}.
do_flow([], InCReq0, InState0) ->
    {ok, InCReq0, InState0};
do_flow([F | Rest], InCReq0, InState0) ->
    case F(InCReq0, InState0) of
        {continue, OutCReq, OutState} ->
            do_flow(Rest, OutCReq, OutState);
        {stop, OutCReq, OutState} ->
            {ok, OutCReq, OutState}
    end.

3 Likes

The readability is the flattening aspect of it however, not the implicit argument positioning (which I honestly quite dislike, I prefer a visible marker).

I’m still ambivalent on it though. It’s a nice-to-have but not really necessary here.

3 Likes

I just want to clarify some things since some things were bent and misinterpreted, and then I’d like to refocus the discussion on actual comparisons.

When I said “It seems to me that it could be added to Erlang with little fanfare.”, I did not say anything about it not being hard to do or anything about the implementation. What I meant, which should have been asked about if confused, is that from my perspective, the pipe operator could be added to the language with little to no impact of the syntax and semantics of the language outside of the pipe operator itself and a pipe output placement operator. In that it being added would have practically zero effect on existing code and future code if one chose to ignore the existence of the pipe operator. It is certainly an open question and the point of the discussion about whether it would be a clarifying addition or a potential distractor.

The fact that some languages don’t have it is not a good argument, because I rarely see those languages, such as Java, Python, C++, as being languages that are held up as shining examples of simplicity, clarity, and consistency and they have little in common with Erlang. The languages I mentioned, such as F#, Clojure, etc. share a lot of similarities with Elixir and Erlang, which was part of the point. Again, this discussion is not just because those languages have a pipe operator. I only used them as an example in passing to illustrate that pipelines are not some controversial thing trying to be slapped onto Erlang just because of other languages. It was just that those languages found it an important feature, so let’s ask if it could be useful in Erlang. I just wanted to explore and ask about it and discover whether it would be a good fit or even possible. I haven’t seen a discussion about it anywhere, which is why I asked here.

Lastly, there are several dissenting arguments but little in the way of actual comparisons between code with and without a pipe operator, and I suppose the same holds true for supporting arguments. Examples from other languages F# and Elixir are not enough, so we should focus on Erlang examples. I think it would be nice to see some examples of how it does or doesn’t help to help focus the discussion, instead of it teetering on rejecting even a discussion of a change just because and for ideological reasons. It’s best to focus on concrete examples and comparisons.


Focusing the discussion

From what I can tell, it is what I described in (2) here being commonly found in Erlang code. I see this everywhere inside the Cowboy repo. The general outline is:

 output_f = f(x),
 output_g = g(output_f),
 output_h = h(output_g),

This pattern may be required in some situations where the pipe operator is not applicable. This actually happens a decent amount in Elixir and F# as well. For example, pipelines don’t handle this well and neither do nested functions:

 output_f = f(x),
 output_g = g(output_f),
 output_h = h(output_f, output_g),

In my opinion, the only available option or hope in Erlang to make a pipe operator work and to be as consistent as possible with both existing code and new code moving forward would be to have a pipe operator |> and a single pipe output placement operator &. From my point of view, the pipeline operator is a great tool that helps clarify situations like the first one above, where data flows through various processing functions. and removes the need to create temporary variables only to pass them on to the next function and avoid nesting function calls. Obviously, even in languages that have the |> operator, one must be careful not to abuse it and apply it unnecessarily.

This completely removes what F# handles by having currying and Elixir solves by inserting pipe outputs into the next pipeline function’s first argument. It also keeps the arity of functions visible. I do not think multiple placement operators should even be considered, and I never proposed that. At the end of the day, this means it’s completely equivalent to nested functions and mostly equivalent to temporary variable bindings. (I say mostly because there are some cases in which previous variables are used that are not the directly preceding variable binding, as illustrated above.) If one accepts the hypothetical of adding the operator, I think the biggest syntactical question is that it sort of has some conflicts with , at the end of expressions.

I think what would be nice are some examples. I’m going to look into some existing Erlang code and see for myself. Having written a fair amount of F# and Elixir, I am a fan of the pipeline operator as it keeps code clear, consistent, and readable in those languages when applicable and used appropriately. However, while I think it’s good to discuss adding it to Erlang, I suppose I myself am not yet convinced, which is why I asked in the first place. One reason I am not sure is that I haven’t developed in Erlang enough to have a mental snapshot of what industrial code looks like. Another reason I am not convinced is that F# and Elixir were designed with the pipe operator in place from the beginning. It may be the case that Erlang code is too inconsistent regarding argument order such that a pipeline operator would be a potentially distracting feature. But let’s look at some comparisons to see, because that’s the crux of the issue. At present, I’m not convinced either way.

3 Likes

Hi Jose! Thank you for weighing in!

I am somewhat surprised by that given Elixir’s heavy use of placement operators like &1, &2, etc. in anonymous functions and function passing, which I in general dislike in favor of the more traditional fn a, b -> ____ end. (E.g., here), and here.) Although, I’m slowly getting sold on them for certain cases that reduce boilerplate. But this is a distraction. Apologies! :stuck_out_tongue:

I think you bring up a good point, and I can imagine some possibilities of confusion if the single pipe output placement operator & “bounces” around in function calls, playing a sort of argument pachinko game.

If I try it:

function_a(Arg),
|> function_b(&, 234),
|> function_c(atom, 123, &),
|> function_d(whatever, &, "abc")

I think this is where some concrete examples from Erlang would be useful to see. I feel the abstract example I provided here isn’t enough to decide. It actually looks okay to me here, but without actual context of function names and arguments, I don’t think the example has enough meat to chew on.

3 Likes

For what it’s worth, what Joe Armstrong actually said regarding |> was:

This is one of those things that is really really good, and really really easy to understand so nobody will give you credit for it.

This is very useful. Suppose we want to capitalize an atom that is turn the atom ‘abc’ into ‘Abc’. There is no function to capitalize an atom in Elixir but there is a function to capitalize a string. So we have to first convert the atom to a string. So in Erlang we’d write

list_to_atom(binary_to_list(capitalize_binary(list_to_binary(atom_to_list(X))))).

Which is horrible. We could also write

capitalize_atom(X) ->
    V1 = atom_to_list(X),
    V2 = list_to_binary(V1),
    V3 = capitalize_binary(V2),
    V4 = binary_to_list(V3),
    binary_to_atom(V4).

Which is even worse - yucky code. How many times have I written code like this? this sucks big time.

With the |> operator this becomes:

X |> atom_to_list |> list_to_binary |> capitalize_binary 
  |> binary_to_list |> binary_to_atom

But again, I think examples are what we’re after. (Like this one but also with more arguments in play.)

4 Likes

I stand corrected. In a very eloquent way, I might add.

FWIW, my aversion to the pipe would be much mitigated if it were observable. This would give it a tangible, as opposed to purely cosmetic, advantage over the nested function calls, without the need for “yucky code” (which is itself a tradeoff of immutability).

A contrived example based on the above, where I am arbitrarily choosing to say that the pipe operator on its own returns a list of the individual steps as it were an alias for [V1, V2, V3, V4, Vn]:

Eshell V12.1.3  (abort with ^G)
1> X |> atom_to_list
1>   |> list_to_binary
1>   |> capitalize_binary 
1>   |> binary_to_list
1>   |> binary_to_atom.
'ERLANG'
2> |>.
["Erlang", ..., ..., "ERLANG", 'ERLANG']

This is purely for illustrative purposes, so please don’t consider it a suggested syntax. I’d be very interested to learn if something akin to this is feasible in Elixir.

3 Likes

Joe certainly a very conversational way of writing and speaking, one I’m envious of.

I am personally not aware of anything like that in Elixir, but I could see that being useful in the same way that being able to print out current bindings would be useful. I think Elixir has a way to print out all current bindings (I would assume this comes from Erlang?). For pipes, a common way that I know of to debug them is to place an IO.inspect() with an optional label argument. It passes through whatever is given to it, but prints the value out to the console.

In Elixir,

%{"a" => 1, "b" => 2}
|> Map.put("c", 3)
|> IO.inspect(label: "Map value")
|> Enum.map(fn {k,v} -> {k,2*v} end)
|> IO.inspect(label: "Keyword list value")
|> Enum.into(%{})

Would print

Map value: %{"a" => 1, "b" => 2, "c" => 3}
Keyword list value: [{"a", 2}, {"b", 4}, {"c", 6}]

to the console and return

%{"a" => 2, "b" => 4, "c" => 6}
2 Likes

Even OCaml has a pipeline operator(|>), and is sound with the notion of functions in there, as all functions are single-arity… which I don’t think is the case with Erlang.

2 Likes

Yes, but then you have to remember you are opening yourself to the possibility of seeing code like this:

function_a(Arg)
|> function_b(&, 234)
|> function_c(
    atom,
    123
    |> function_c1(&, foo)
    |> function_c2(bar, &),
    &
)
|> function_d(whatever, &, "abc")

Which is already debatable in Elixir but it does cross the line for me with the & notation.

And should this be allowed too?

function_a(Arg)
|> function_b(& + 1, 234)

And, if yes, how does it compose with the nesting of the previous example? And there are probably other corner cases that would appear too.

6 Likes