Buffering of stdin and stderr when doing open_port and how to change it

I have a external executable on Linux which uses stdout and stderr to send us error and info messages.

We would like to have them line buffered and available immediately to the Erlang process without further buffering.

Our current call is:

    Port = open_port({spawn_executable, Path},
                     [use_stdio,
                      stderr_to_stdout,
                      binary,
                      {line, 80},
                      {args, Args}])

The {line 80} takes care of the Erlang side trying to immediately send us completed lines (or parts of too long ones).

What we observe however seems to be not line buffered but fully buffered i.e. it takes a while and then we get many lines at once.

There are multiple sources of buffering here:

  1. Output buffering in the executable, lets assume for now thats not the case (but maybe there is a bug)
  2. stdin being fully buffered when reading from a pipe
  3. stderr shouldn’t bebuffered but I don’t know what the stderr_to_stdout does to that.
  4. Tty buffering settings: I’m not assuming they apply since we just have a pipe.

How can I actually get line buffered behaviour here (assuming the executable behaves)?
Or how can we find out that the exeuctable is the cause of buffering?

I used to know such stuff but its been a while…:wink:

2 Likes

Use stdbuf -oL maybe?

But isn’t that just changing the buffering on the output side where our current assumption is it’s done correctly?

Yes it is. Is that assumption correct?

Some programs configure their buffering based on whether their output is a TTY. An Erlang port isn’t a TTY, so the buffering might be different. stdbuf -oL attempts to force it back into line-buffering.