What do you think should be in OTP?

Chat / Discussions
,
102

I look at the BERT spec. And I see that “BERT … is a … binary data interchange format based on (and compatible with) Erlang’s binary serialization”. But this spec knows nothing about maps for example:)

1 Like
103

What can I say - nothing is perfect. But - any specification can be tried to expand :wink:.

104

I agree. But we already have full specification of Erlang’s binary serialization format. And we already have set of many languages libraries for serialization/deserialization this format (thanks to Michael Truog).

105

That is very non-engineering reasoning there. JSON is okayish format if you want:

  • machine to machine communication in more or less uniform way
  • representation should be fairly concise
  • representation should be readable and editable by human operator

If we take these points into consideration, then JSON do not really have any reasonable competition. Any competitor breaks at least of the above points.

Is JSON perfect? No, but any other format is not perfect either.

What is wrong with Rebar3?

Preprocessor is capable of that.

I worked (the work stalled a little) on defining safe subset of the ETF that could be used between different languages, and it is far from perfect:

  • There is no single way to encode boolean
  • If you want encode atoms, then you have to implement 2 forms of encoding (atoms table and inline) and you need to implement a way to safely decode them (to prevent atom exhaustion as atoms aren’t GCed)
  • You need to prevent encoding some data, like functions, as these can impose the security threat
  • You cannot encode all IEEE 754 floats range
  • Floats can be stored as strings or in binary representation
  • Integers are stored in network order (aka big endian) which slows down decoding on majority of CPUs (as you need to swap bytes before use, you cannot decode them “in place”)
  • Encoding non-ASCII Erlang strings is costly (as these are encoded as lists and cannot use the STRING_EXT), and single Unicode character forces whole string to be encoded in the “bloated” form
  • Encoding 32-bit unsigned integer require 2x more space than encoding 32-bit signed integer

If you want to have cross-platform and safe encoding format, then ETF is not something that you should look for.

3 Likes
106

Great points!

Another thing I love about JSON is that every language has a parser. MessagePack, cbor, and especially term_to_binary are comparitively rare and I can’t depend on it being usable.

2 Likes
107

I found the Stream module in Elixir useful. Pipes too, but I’m not sure if that easily fits in to idiomatic Erlang.

1 Like
108

Wow folks! Just check it out! The JSON idea from this topic already prepared for Erlang/OTP - Add json module by michalmuskala · Pull Request #8111 · erlang/otp · GitHub! Amazing! What can be next?

8 Likes