# Euneus - A JSON parser and generator in pure Erlang

**URL:** <https://erlangforums.com/t/euneus-a-json-parser-and-generator-in-pure-erlang/3031>\
**Category:** Libraries\
**Tags:** json, euneus\
**Created:** [November 14, 2023, 9:29pm UTC](https://erlangforums.com/t/euneus-a-json-parser-and-generator-in-pure-erlang/3031 "2023-11-14T21:29:32Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![williamthome](https://erlangforums.com/user_avatar/erlangforums.com/williamthome/32/471_2.png) [@williamthome](https://erlangforums.com/u/williamthome)\
**Post date:** [November 14, 2023, 9:29pm UTC](https://erlangforums.com/t/euneus-a-json-parser-and-generator-in-pure-erlang/3031/1 "2023-11-14T21:29:32Z")

</div>

# Euneus

> **[GitHub - williamthome/euneus: An incredibly flexible and performant JSON parser,...](https://github.com/williamthome/euneus)**
>
> An incredibly flexible and performant JSON parser, generator and formatter in pure Erlang.

## What is Euneus?

A JSON parser and generator in pure Erlang. Euneus is a rewrite of [Thoas](https://github.com/lpil/thoas). Like Thoas, both the parser and generator fully conform to [RFC 8259](https://tools.ietf.org/html/rfc8259) and [ECMA 404](http://www.ecma-international.org/publications/standards/Ecma-404.htm).

## Why Euneus over Thoas?

Thoas is incredible, works performant and perfectly fine, but Euneus is more flexible, permitting more customizations. The motivation for Euneus is [this PR](https://github.com/lpil/thoas/pull/28) when I was looking for options to not traverse lists and maps to customize terms.

## How does it perform?

> **Note**
> 
> - [Jason](https://github.com/michalmuskala/jason) it’s an Elixir library that Thoas was based on.
> - See more benchmarks [here](https://github.com/williamthome/euneus?tab=readme-ov-file#benchmarks).

### Encode

 ![eunes-encode-ips](https://erlangforums.com/uploads/default/original/2X/9/94f6a0096a46f3a2363c18a822efac5caa992d2c.png)

 ![euneus-encode-memory](https://erlangforums.com/uploads/default/original/2X/a/a64b12b9b6019aa7534e98e63df7cd369b90836e.png)

```erlang
##### With input Blockchain #####
Name ips average deviation median 99th %
euneus 10.56 K 94.69 μs ±15.18% 95.53 μs 139.58 μs
Jason 10.35 K 96.58 μs ±16.88% 85.48 μs 155.88 μs
thoas 7.42 K 134.68 μs ±9.91% 135.49 μs 175.04 μs

Comparison: 
euneus 10.56 K
Jason 10.35 K - 1.02x slower +1.89 μs
thoas 7.42 K - 1.42x slower +39.99 μs

Memory usage statistics:

Name Memory usage
euneus 83.10 KB
Jason 78.91 KB - 0.95x memory usage -4.19531 KB
thoas 89.41 KB - 1.08x memory usage +6.31 KB

```

### Decode

 ![euneus-decode-ips](https://erlangforums.com/uploads/default/original/2X/1/12eb9ef2ad4ddd8cae26c712c9061e37088a4be7.png)

 ![euneus-decode-memory](https://erlangforums.com/uploads/default/original/2X/5/5bd452fa069e5ca3cf881ec3bb3bc9dd9f142ead.png)

```erlang
##### With input Blockchain #####
Name ips average deviation median 99th %
euneus 6.98 K 143.35 μs ±5.22% 141.99 μs 178.33 μs
Jason 6.93 K 144.26 μs ±14.21% 140.26 μs 274.38 μs
thoas 5.65 K 177.01 μs ±8.87% 175.34 μs 252.45 μs

Comparison: 
euneus 6.98 K
Jason 6.93 K - 1.01x slower +0.91 μs
thoas 5.65 K - 1.23x slower +33.66 μs

Memory usage statistics:

Name Memory usage
euneus 51.41 KB
Jason 51.63 KB - 1.00x memory usage +0.21 KB
thoas 51.41 KB - 1.00x memory usage +0 KB

```

## Installation

Euneus is available at [Hex](https://hex.pm/packages/euneus).

### Erlang

```erlang
% rebar.config
{deps, [{euneus, "0.7.0"}]}

```

### Elixir

```elixir
def deps do
  [{:euneus, "~> 0.7"}]
end

```

## Customizations

> Please see the [README](https://github.com/williamthome/euneus?tab=readme-ov-file#differences-to-thoas) file in the repository for more info.

### Encode

```erlang
#{
    %% nulls defines what terms will be replaced with the null literal (default: ['undefined']).
    nulls => nonempty_list(),
    %% binary_encoder allow override the binary() encoding.
    binary_encoder => function((binary(), euneus_encoder:options()) -> iolist()),
    %% atom_encoder allow override the atom() encoding.
    atom_encoder => function((atom(), euneus_encoder:options()) -> iolist()),
    %% integer_encoder allow override the integer() encoding.
    integer_encoder => function((integer(), euneus_encoder:options()) -> iolist()),
    %% float_encoder allow override the float() encoding.
    float_encoder => function((float(), euneus_encoder:options()) -> iolist()),
    %% list_encoder allow override the list() encoding.
    list_encoder => function((list(), euneus_encoder:options()) -> iolist()),
    %% map_encoder allow override the map() encoding.
    map_encoder => function((map(), euneus_encoder:options()) -> iolist()),
    %% datetime_encoder allow override the calendar:datetime() encoding.
    datetime_encoder => function((calendar:datetime(), euneus_encoder:options()) -> iolist()),
    %% timestamp_encoder allow override the erlang:timestamp() encoding.
    timestamp_encoder => function((erlang:timestamp(), euneus_encoder:options()) -> iolist()),
    %% unhandled_encoder allow encode any custom term (default: raise unsupported_type error).
    unhandled_encoder => function((term(), euneus_encoder:options()) -> iolist()),
    %% escaper allow override the binary escaping (default: json)
    escaper => json
             | html
             | javascript
             | unicode
             | function((binary(), euneus_encoder:options()) -> iolist()),
    error_handler => function(( error | exit | throw
                              , term()
                              , erlang:stacktrace() ) -> euneus_encoder:result())
}

```

### Decode

```erlang
#{
    %% null_term is the null literal override (default: 'undefined').
    null_term => term(),
    %% arrays allow override any array/list().
    arrays => function((list(), euneus_decoder:options()) -> term()),
    %% objects allow override any object/map().
    objects => function((map(), euneus_decoder:options()) -> term()),
    %% keys allow override the keys from JSON objects.
    keys => copy
          | to_atom
          | to_existing_atom
          | to_integer
          | function((binary(), euneus_decoder:options()) -> term()),
    %% values allow override any other term, like array item or object value.
    values => copy
            | to_atom
            | to_existing_atom
            | to_integer
            | function((binary(), euneus_decoder:options()) -> term())
}

```

#### Decode resuming

Euneus permits resuming the decoding when an invalid token is found. Any value can replace the invalid token by overriding the `error_handler` option. Please see an example [here](https://github.com/williamthome/euneus?tab=readme-ov-file#resuming).

## Credits

Euneus is a rewrite of Thoas, so all credits go to [Michał Muskała](https://github.com/michalmuskala), [Louis Pilfold](https://github.com/lpil), also both [Jason](https://github.com/michalmuskala/jason) and [Thoas](https://github.com/lpil/thoas) contributors. Thanks for the hard work!

---

<div class="post-metadata">

**Author:** ![williamthome](https://erlangforums.com/user_avatar/erlangforums.com/williamthome/32/471_2.png) [@williamthome](https://erlangforums.com/u/williamthome)\
**Post date:** [November 16, 2023, 4:43pm UTC](https://erlangforums.com/t/euneus-a-json-parser-and-generator-in-pure-erlang/3031/2 "2023-11-16T16:43:15Z")

</div>

# New release

`v1.0.0` is released with a plugin mechanism implemented.

## Changes

A mechanism to easily plug in encoders and decoders is now implemented. You can use the [built-in plugins](https://github.com/williamthome/euneus?tab=readme-ov-file#built-in-plugins) to handle common types or create your own in a module by implementing the [euneus\_plugin](https://github.com/williamthome/euneus/blob/9e37e248cd384e6264536c47cb316e2c6fc5adb2/src/euneus_plugin.erl#L22) behavior.

## Plugins Usage

```erlang
1> {ok, JSON} = euneus:encode_to_binary({127,0,0,1}, #{plugins => [inet]}).
{ok,<<"\"127.0.0.1\"">>}

2> euneus:decode(JSON, #{plugins => [inet]}).
{ok,{127,0,0,1}}

```

## Built-in Plugins

Currently, these are the built-in plugins:

- [datetime](https://github.com/williamthome/euneus?tab=readme-ov-file#datetime)
- [inet](https://github.com/williamthome/euneus?tab=readme-ov-file#inet)
- [pid](https://github.com/williamthome/euneus?tab=readme-ov-file#pid)
- [port](https://github.com/williamthome/euneus?tab=readme-ov-file#port)
- [proplist](https://github.com/williamthome/euneus?tab=readme-ov-file#proplist)
- [reference](https://github.com/williamthome/euneus?tab=readme-ov-file#reference)
- [timestamp](https://github.com/williamthome/euneus?tab=readme-ov-file#timestamp)

### Data Mapping

```erlang
| **Erlang ->** | **Encode Options ->** | **JSON ->** | **Decode Options ->** | **Erlang** |
|--------------------------------------------------------------	|---------------------------	|--------------------------------------	|---------------------------	|--------------------------------------------------------------	|
| {{1970,1,1},{0,0,0}} | #{plugins => [datetime]} | "1970-01-01T00:00:00Z" | #{plugins => [datetime]} | {{1970,1,1},{0,0,0}} |
| {127,0,0,1} | #{plugins => [inet]} | "127.0.0.1" | #{plugins => [inet]} | {127,0,0,1} |
| {16#3ffe,16#b80,16#1f8d,16#2,16#204,16#acff,16#fe17,16#bf38} | #{plugins => [inet]} | "3ffe:b80:1f8d:2:204:acff:fe17:bf38" | #{plugins => [inet]} | {16#3ffe,16#b80,16#1f8d,16#2,16#204,16#acff,16#fe17,16#bf38} |
| <0.92.0> | #{plugins => [pid]} | "<0.92.0>" | #{plugins => [pid]} | <0.92.0> |
| #Port<0.1> | #{plugins => [port]} | "#Port<0.1>" | #{plugins => [port]} | #Port<0.1> |
| [{foo, bar}] | #{plugins => [proplist]} | {\"foo\":\"bar\"} | #{} | #{<<"foo">> => <<"bar">>} |
| #Ref<0.957048870.857473026.108035> | #{plugins => [reference]} | "#Ref<0.957048870.857473026.108035>" | #{plugins => [reference]} | #Ref<0.957048870.857473026.108035> |
| {0,0,0} | #{plugins => [timestamp]} | 1970-01-01T00:00:00.000Z | #{plugins => [timestamp]} | {0,0,0} |

```

## Custom Plugins

Custom plugins must be implemented using the [euneus\_plugin](https://github.com/williamthome/euneus/blob/9e37e248cd384e6264536c47cb316e2c6fc5adb2/src/euneus_plugin.erl#L22) behavior, for example:

```erlang
-module(euneus_test_plugin).
-behaviour(euneus_plugin).
-export([encode/2, decode/2]).

encode({test, foo}, Opts) ->
    {halt, euneus_encoder:encode_binary(<<"test::foo">>, Opts)};
encode(_Term, _Opts) ->
    next.

decode(<<"test::foo">>, _Opts) ->
    {halt, {test, foo}};
decode(_Bin, _Opts) ->
    next.

```

```erlang
1> {ok, JSON} = euneus:encode_to_binary({test, foo}, #{plugins => [euneus_test_plugin]}).
{ok,<<"\"test::foo\"">>}

2> euneus:decode(JSON, #{plugins => [euneus_test_plugin]}).
{ok,{test, foo}}

```

## Deprecation

The `datetime_encoder` and the `timestamp_encoder` options are now deprecated in favor of the [datetime](https://github.com/williamthome/euneus?tab=readme-ov-file#datetime) and [timestamp](https://github.com/williamthome/euneus?tab=readme-ov-file#timestamp) plugins.

## Installation

### Erlang

```erlang
% rebar.config
{deps, [{euneus, "1.0.1"}]}

```

### Elixir

```erlang
def deps do
  [{:euneus, "~> 1.0"}]
end

```

## Note

If you have a built-in plugin suggestion, feel free to open a [new issue](https://github.com/williamthome/euneus/issues/new) to discuss it.

---

<div class="post-metadata">

**Author:** ![williamthome](https://erlangforums.com/user_avatar/erlangforums.com/williamthome/32/471_2.png) [@williamthome](https://erlangforums.com/u/williamthome)\
**Post date:** [November 18, 2023, 3:44am UTC](https://erlangforums.com/t/euneus-a-json-parser-and-generator-in-pure-erlang/3031/3 "2023-11-18T03:44:59Z")

</div>

# New release

Version [1.0.2](https://github.com/williamthome/euneus/releases/tag/v1.0.2) was released with a great improvement in the plugin mechanism and a bunch of benchmark runs and comparisons.

Please see the [benchmarks](https://github.com/williamthome/euneus?tab=readme-ov-file#benchmarks) in the README file for more info.  
All runs summaries are in there by comparing:

- [euneus](https://github.com/williamthome/euneus)
- [thoas](https://github.com/lpil/thoas)
- [jsone](https://github.com/sile/jsone)
- [jsx](https://github.com/talentdeficit/jsx)
- [jiffy](https://github.com/davisp/jiffy)
- [Jason](https://github.com/michalmuskala/jason)

All the benchmarks details are available [here](https://github.com/williamthome/euneus/tree/main/euneus_bench/runs).

## Edit

I fixed the decode benchmarks and improved the readability of the benchmarks.  
The decode benchmark with parsed options was erroneously copied from the decode with plugins benchmarks.

---

<div class="post-metadata">

**Author:** ![williamthome](https://erlangforums.com/user_avatar/erlangforums.com/williamthome/32/471_2.png) [@williamthome](https://erlangforums.com/u/williamthome)\
**Post date:** [November 20, 2023, 2:03am UTC](https://erlangforums.com/t/euneus-a-json-parser-and-generator-in-pure-erlang/3031/4 "2023-11-20T02:03:50Z")

</div>

# New release

Version `1.1.0` is out with format functions.

## Changes

A model called `euneus_formatter` is implemented with three major functions:

- minify
- prettify
- format

## Usage

### Minify

```erlang
1> iolist_to_binary(euneus:minify(<<"{\n \"foo\": \"bar\",\n \"baz\": {\n \"foo\": \"bar\",\n \"0\": [\n \"foo\",\n 0\n]\n }\n}">>)).
<<"{\"foo\":\"bar\",\"baz\":{\"foo\":\"bar\",\"0\":[\"foo\",0]}}">>

```

### Prettify

```erlang
1> io:format("~s~n", [euneus:prettify(<<"{\"foo\":\"bar\",\"baz\":{\"foo\":\"bar\",\"0\":[\"foo\",0]}}">>)]).
{
  "foo": "bar",
  "baz": {
    "foo": "bar",
    "0": [
      "foo",
      0
    ]
  }
}

```

### Custom

```erlang
1> Opts = #{spaces => <<$\t>>, indent => <<$\t, $\t>>, crlf => <<$\n>>}.

2> io:format("~s~n", [euneus:format(<<"{\"foo\":\"bar\",\"baz\":{\"foo\":\"bar\",\"0\":[\"foo\",0]}}">>, Opts)]).
{
                "foo": "bar",
                "baz": {
                                "foo": "bar",
                                "0": [
                                                "foo",
                                                0
                                ]
                }
}

```

---

<div class="post-metadata">

**Author:** ![asabil](https://erlangforums.com/user_avatar/erlangforums.com/asabil/32/381_2.png) [@asabil](https://erlangforums.com/u/asabil)\
**Post date:** [November 20, 2023, 1:39pm UTC](https://erlangforums.com/t/euneus-a-json-parser-and-generator-in-pure-erlang/3031/5 "2023-11-20T13:39:35Z")

</div>

This is awesome, thank you!

Just wondering, given the plugin/extensibility mechanisms, how does one goes about ignoring fields when encoding a map into a JSON object, for example:

```erlang
#{a => 1, b => undefined}

```

would be encode as:

```json
{"a": 1}

```

Would this require overriding the map\_encoder altogether? Either way, an example would be greatly appreciated.

Thanks

---

<div class="post-metadata">

**Author:** ![LeonardB](https://erlangforums.com/user_avatar/erlangforums.com/leonardb/32/1856_2.png) [@LeonardB](https://erlangforums.com/u/LeonardB)\
**Post date:** [November 20, 2023, 2:14pm UTC](https://erlangforums.com/t/euneus-a-json-parser-and-generator-in-pure-erlang/3031/6 "2023-11-20T14:14:34Z")

</div>

I don’t think the plugin or encoder system currently supports a skip/drop return, but I think that would be nice/useful to have.

Currently the spec is

```erlang
-callback encode(Input, Opts) -> Output when
Input :: term(),
Opts :: euneus_encoder:options(),
Output :: {halt, iolist()} | next.

-callback decode(Input, Opts) -> Output when
Input :: term(),
Opts :: euneus_decoder:options(),
Output :: {halt, term()} | next.

```

Adding support for atom `skip` or `drop` to allow discarding of the key/value would make things more flexible.

A simple use case would be dropping properties with null values.

The default behavior is to encode `undefined` as `null` though

---

<div class="post-metadata">

**Author:** ![williamthome](https://erlangforums.com/user_avatar/erlangforums.com/williamthome/32/471_2.png) [@williamthome](https://erlangforums.com/u/williamthome)\
**Post date:** [November 20, 2023, 2:16pm UTC](https://erlangforums.com/t/euneus-a-json-parser-and-generator-in-pure-erlang/3031/7 "2023-11-20T14:16:08Z")

</div>

Thanks!

@paulo-f-oliveira also asked me about this.

TLDR, yes, it’s possible, e.g.:

```erlang
-module(euneus_ignore_undefined_keys_plugin).
-behaviour(euneus_plugin).
-export([encode/2, decode/2]).

encode(Map0, Opts) when is_map(Map0) ->
    Map = maps:filter(fun(_, V) -> V =/= undefined end, Map0),
    {halt, euneus_encoder:encode_map(Map, Opts)};
encode(_Term, _Opts) ->
    next.

decode(_Bin, _Opts) ->
    next.

```

```erlang
1> euneus:encode_to_binary(#{a => 1, b => undefined}, #{plugins => [euneus_ignore_undefined_keys_plugin]}).
{ok,<<"{\"a\":1}">>}

```

Maybe worth having a built-in plugin for this? It will be a bit more performant.

## Edit

Maybe using the ‘nulls’ option can be better:

```erlang
encode(Map0, Opts) when is_map(Map0) ->
    % By default, the 'nulls' option is [undefined].
    Nulls = euneus_encoder:get_nulls_option(Opts),
    Map = maps:filter(fun(_, V) -> not lists:member(V, Nulls) end, Map0),
    {halt, euneus_encoder:encode_map(Map, Opts)};
encode(_Term, _Opts) ->
    next.

```

---

<div class="post-metadata">

**Author:** ![LeonardB](https://erlangforums.com/user_avatar/erlangforums.com/leonardb/32/1856_2.png) [@LeonardB](https://erlangforums.com/u/LeonardB)\
**Post date:** [November 20, 2023, 2:20pm UTC](https://erlangforums.com/t/euneus-a-json-parser-and-generator-in-pure-erlang/3031/8 "2023-11-20T14:20:57Z")

</div>

Having it as a default would be great. Would it maybe make more sense to just be an option flag though?

`drop_nulls` or something like that?

That perform better since it could be a clean function head match.

---

<div class="post-metadata">

**Author:** ![asabil](https://erlangforums.com/user_avatar/erlangforums.com/asabil/32/381_2.png) [@asabil](https://erlangforums.com/u/asabil)\
**Post date:** [November 20, 2023, 2:25pm UTC](https://erlangforums.com/t/euneus-a-json-parser-and-generator-in-pure-erlang/3031/9 "2023-11-20T14:25:27Z")

</div>

Actually, one important aspect for me is to make a clear distinction between `null` and `undefined` in the same way that JavaScript does.

```javascript
> JSON.stringify({a: 1, b: undefined})
'{"a":1}'
> JSON.stringify({a: 1, b: null})
'{"a":1,"b":null}'

```

---

<div class="post-metadata">

**Author:** ![williamthome](https://erlangforums.com/user_avatar/erlangforums.com/williamthome/32/471_2.png) [@williamthome](https://erlangforums.com/u/williamthome)\
**Post date:** [November 20, 2023, 2:36pm UTC](https://erlangforums.com/t/euneus-a-json-parser-and-generator-in-pure-erlang/3031/10 "2023-11-20T14:36:44Z")

</div>

I’m thinking of changing the encode and decode functions of the `eunues_plugin` behavior to receive any argument/plugin options:

```erlang
-module(euneus_plugin).

-callback encode(Input, PluginOpts, EncodeOpts) -> Output when
    Input :: term(),
    PluginOpts :: term(),
    EncodeOpts :: euneus_encoder:options(),
    Output :: {halt, iolist()} | next.

-callback decode(Input, PluginOpts, DecodeOpts) -> Output when
    Input :: term(),
    PluginOpts :: undefined | term()
    DecodeOpts :: euneus_decoder:options(),
    Output :: {halt, term()} | next.

```

So plugins can be defined as the name or a tuple of 2 elements where the first would be the name and the second one the plugin options: `#{plugins => [plugin_name]}` or `#{plugins => [{plugin_name, plugin_options}]`. If only the name is passed, maybe the default plugin options be ‘undefined’ or an empty list or map.  
In this way, my example to ignore keys can be:

```erlang
encode(Map0, Terms, Opts) when is_map(Map0) ->
    Map = maps:filter(fun(_, V) -> not lists:member(V, normalize_terms(Terms)) end, Map0),
    {halt, euneus_encoder:encode_map(Map, Opts)};
encode(_Term, _Opts) ->
    next.

normalize_terms(Terms) when is_list(Terms) -> 
    Terms;
normalize_terms(Term) -> 
    [Term].

```

```erlang
1> euneus:encode_to_binary(#{a => 1, b => undefined}, #{plugins => [{euneus_ignore_undefined_keys_plugin, [undefined]}]}).
{ok,<<"{\"a\":1}">>}

```

I do not have tested/implemented it, just an idea.  
What do you think?

## Edit

There is a `nulls` option for the encode function, it is a list, so my first example can be:

```erlang
encode(Map0, Opts) when is_map(Map0) ->
    % By default, the 'nulls' option is [undefined].
    Nulls = euneus_encoder:get_nulls_option(Opts),
    Map = maps:filter(fun(_, V) -> not lists:member(V, Nulls) end, Map0),
    {halt, euneus_encoder:encode_map(Map, Opts)};
encode(_Term, _Opts) ->
    next.

```

## Edit 2

Just pushed a draft [PR](https://github.com/williamthome/euneus/pull/35) implementing the `drop_nulls` plugin.

---

<div class="post-metadata">

**Author:** ![williamthome](https://erlangforums.com/user_avatar/erlangforums.com/williamthome/32/471_2.png) [@williamthome](https://erlangforums.com/u/williamthome)\
**Post date:** [November 20, 2023, 6:46pm UTC](https://erlangforums.com/t/euneus-a-json-parser-and-generator-in-pure-erlang/3031/11 "2023-11-20T18:46:54Z")

</div>

# New release

Version [1.2.0](https://github.com/williamthome/euneus/releases/tag/v1.2.0) is released with a new built-in plugin called `drop_nulls` that drops all keys of maps and proplists that have the value included in the `nulls` list of the encoding option.

## `drop_nulls` example

### Maps

```erlang
1> euneus:encode_to_binary(#{a => 1, b => undefined}, #{plugins => [drop_nulls]}).
{ok,<<"{\"a\":1}">>}

2> euneus:encode_to_binary(#{a => 1, b => undefined, c => nil}, #{nulls => [undefined, nil], plugins => [drop_nulls]}).
{ok,<<"{\"a\":1}">>}

```

### Proplists

```erlang
1> euneus:encode_to_binary([{a, 1}, {b, undefined}], #{plugins => [proplist, drop_nulls]}).
{ok,<<"{\"a\":1}">>}

2> euneus:encode_to_binary([{a, 1}, {b, undefined}, {c, nil}], #{nulls => [undefined, nil], plugins => [proplist, drop_nulls]}).
{ok,<<"{\"a\":1}">>}

```

## Installation

### Erlang

```erlang
% rebar.config
{deps, [{euneus, "1.2.0"}]}

```

### Elixir

```erlang
# mix.exs
def deps do
  [{:euneus, "~> 1.2"}]
end

```

## Credits

Thanks @asabil, @LeonardB and also @paulo-f-oliveira!

---

<div class="post-metadata">

**Author:** ![saleyn](https://erlangforums.com/user_avatar/erlangforums.com/saleyn/32/1188_2.png) [@saleyn](https://erlangforums.com/u/saleyn)\
**Post date:** [November 21, 2023, 12:40am UTC](https://erlangforums.com/t/euneus-a-json-parser-and-generator-in-pure-erlang/3031/12 "2023-11-21T00:40:06Z")

</div>

I added `euneus` to JSON parser benchmarks at [GitHub - saleyn/simdjsone: Erlang Fast JSON parser](https://github.com/saleyn/simdjsone)

```erlang
=== Benchmark (file size: 616.7K) ===
   simdjsone: 5539.670us
      euneus: 8435.540us
       thoas: 8902.160us
       jiffy: 13688.250us

=== Benchmark (file size: 1.3K) ===
   simdjsone: 8.030us
       jiffy: 14.950us
       thoas: 14.960us
      euneus: 19.830us

=== Benchmark (file size: 0.1K) ===
   simdjsone: 1.530us
       jiffy: 2.700us
      euneus: 3.220us
       thoas: 3.600us

```

---

<div class="post-metadata">

**Author:** ![williamthome](https://erlangforums.com/user_avatar/erlangforums.com/williamthome/32/471_2.png) [@williamthome](https://erlangforums.com/u/williamthome)\
**Post date:** [November 21, 2023, 4:26am UTC](https://erlangforums.com/t/euneus-a-json-parser-and-generator-in-pure-erlang/3031/13 "2023-11-21T04:26:56Z")

</div>

That’s nice!

Do you know what’s the reason for this difference in the benchmarks of the [README](https://github.com/saleyn/simdjsone?tab=readme-ov-file#performance-benchmark)?

```erlang
$ make benchmark
=== Benchmark (file size: 1.3K) ===
   simdjsone: 8.030us
       jiffy: 14.950us
       thoas: 14.960us
      euneus: 19.830us

```

VS

```erlang
$ MIX_ENV=test make benchmark
=== Benchmark (file size: 1.3K) ===

Name ips average deviation median 99th %
simdjsone 128.43 K 7.79 μs ±468.75% 5.60 μs 21.10 μs
euneus 106.19 K 9.42 μs ±87.91% 8.80 μs 21.40 μs
poison 97.80 K 10.23 μs ±74.31% 9.30 μs 23 μs
jason 96.92 K 10.32 μs ±98.77% 9.50 μs 26.30 μs
jiffy 90.31 K 11.07 μs ±97.74% 9.20 μs 45.30 μs
thaos 79.04 K 12.65 μs ±133.82% 11.50 μs 26.70 μs

```

FWIW I’m unable to compile and run the benchmarks, I got this error:

```erlang
make nif
make -C c_src
make[1]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule.
make[1]: Entering directory '/home/williamthome/Projects/erlang/simdjsone/c_src'
g++ simdjson.o simdjson_nif.o -L/home/williamthome/Projects/erlang/otp/lib/erl_interface/lib -lei -shared -o /home/williamthome/Projects/erlang/simdjsone/priv/simdjsone.so
/usr/bin/ld: cannot find -lei: No such file or directory
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:84: /home/williamthome/Projects/erlang/simdjsone/priv/simdjsone.so] Error 1
make[1]: Leaving directory '/home/williamthome/Projects/erlang/simdjsone/c_src'
make: *** [Makefile:19: nif] Error 2

```

---

<div class="post-metadata">

**Author:** ![williamthome](https://erlangforums.com/user_avatar/erlangforums.com/williamthome/32/471_2.png) [@williamthome](https://erlangforums.com/u/williamthome)\
**Post date:** [November 21, 2023, 5:37am UTC](https://erlangforums.com/t/euneus-a-json-parser-and-generator-in-pure-erlang/3031/14 "2023-11-21T05:37:49Z")

</div>

Regarding null, please help me by answering my question below in [this issue](https://github.com/williamthome/euneus/issues/36).

> What do you see as the best approach to consider null, undefined, nil, etc from Erlang to JSON and null from JSON to Erlang?

---

<div class="post-metadata">

**Author:** ![saleyn](https://erlangforums.com/user_avatar/erlangforums.com/saleyn/32/1188_2.png) [@saleyn](https://erlangforums.com/u/saleyn)\
**Post date:** [November 21, 2023, 10:02pm UTC](https://erlangforums.com/t/euneus-a-json-parser-and-generator-in-pure-erlang/3031/15 "2023-11-21T22:02:12Z")

</div>

> Do you know what’s the reason for this difference in the benchmarks of the [README](https://github.com/saleyn/simdjsone?tab=readme-ov-file#performance-benchmark)?

The first one is hand-coded in Erlang and executes each test concurrently, and the second is using Elixir’s Benchee. I haven’t tried to compare the results.

> FWIW I’m unable to compile and run the benchmarks, I got this error.

Please provide more info about your environment and maybe submit an issue.

---

<div class="post-metadata">

**Author:** ![nzok](https://erlangforums.com/letter_avatar_proxy/v4/letter/n/4af34b/32.png) [@nzok](https://erlangforums.com/u/nzok)\
**Post date:** [November 22, 2023, 12:15am UTC](https://erlangforums.com/t/euneus-a-json-parser-and-generator-in-pure-erlang/3031/16 "2023-11-22T00:15:09Z")

</div>

Javascript can distinguish between ‘undefined’ and ‘null’, but it tries not to:

> undefined === null  
> false  
> undefined == null  
> true

JSON cannot distinguish between undefined and null. This is one of the reasons why _generic_ conversions between Erlang and JSON can never be perfect. The rules for conversion should be set by a higher-order protocol, and implemented by using a ‘streaming’ interface where the higher-order protocol module generates JSON syntax without ever building a JSON object. Do not trust any “always represent in Erlang as in JSON” advice; it certainly isn’t “best practice”.

---

<div class="post-metadata">

**Author:** ![williamthome](https://erlangforums.com/user_avatar/erlangforums.com/williamthome/32/471_2.png) [@williamthome](https://erlangforums.com/u/williamthome)\
**Post date:** [December 5, 2023, 12:52am UTC](https://erlangforums.com/t/euneus-a-json-parser-and-generator-in-pure-erlang/3031/17 "2023-12-05T00:52:28Z")

</div>

# New release

Version [v1.2.1](https://github.com/williamthome/euneus/releases/tag/v1.2.1) fixes a bug in the decoder’s `values` and `keys` options.

## Credits

Thanks, @michalmuskala, for figuring it out.

---

<div class="post-metadata">

**Author:** ![williamthome](https://erlangforums.com/user_avatar/erlangforums.com/williamthome/32/471_2.png) [@williamthome](https://erlangforums.com/u/williamthome)\
**Post date:** [August 5, 2024, 12:13am UTC](https://erlangforums.com/t/euneus-a-json-parser-and-generator-in-pure-erlang/3031/18 "2024-08-05T00:13:05Z")

</div>

# New release

[Euneus v2.0](https://hex.pm/packages/euneus/2.0.0) is out!

This version completely rewrites Euneus on top of the new OTP json module. The [json\_polyfill](https://hex.pm/packages/json_polyfill) lib is required for OTP versions below 27.

The minimum OTP version is 24.

## Changes

All modules received a new interface.

Many changes were made based on [this issue](https://github.com/williamthome/euneus/issues/36) (many thanks to all contributors o/).

### Encode

Encode functions available:

- [euneus:encode/1](https://hexdocs.pm/euneus/2.0.0/euneus.html#encode/1)
- [euneus:encode/2](https://hexdocs.pm/euneus/2.0.0/euneus.html#encode/2)
- [euneus:encode\_iodata/1](https://hexdocs.pm/euneus/2.0.0/euneus.html#encode_iodata/1)
- [euneus:encode\_iodata/2](https://hexdocs.pm/euneus/2.0.0/euneus.html#encode_iodata/2)

The `euneus_encoder` options:

```erlang
-type options() :: #{
    codecs => [codec()],
    nulls => [term()],
    skip_values => [term()],
    key_to_binary => fun((term()) -> binary()),
    sort_keys => boolean(),
    proplists => boolean() | {true, is_proplist()},
    escape => fun((binary()) -> iodata()),
    encode_integer => encode(integer()),
    encode_float => encode(float()),
    encode_atom => encode(atom()),
    encode_list => encode(list()),
    encode_map => encode(map()),
    encode_tuple => encode(tuple()),
    encode_pid => encode(pid()),
    encode_port => encode(port()),
    encode_reference => encode(reference())
}.

-type codec() ::
    timestamp
    | datetime
    | ipv4
    | ipv6
    | {records, #{Name :: atom() := {Fields :: [atom()], Size :: pos_integer()}}}
    | codec_callback().
-export_type([codec/0]).

-type codec_callback() :: fun((tuple()) -> next | {halt, term()}).

-type is_proplist() :: fun((list()) -> boolean()).

-type encode(Type) :: fun((Type, json:encoder(), state()) -> iodata()).

```

Please see the [encoder documentation](https://hexdocs.pm/euneus/2.0.0/euneus_encoder.html) for further information and examples.

### Decode

Decode functions available:

- [euneus:decode/1](https://hexdocs.pm/euneus/2.0.0/euneus.html#decode/1)
- [euneus:decode/2](https://hexdocs.pm/euneus/2.0.0/euneus.html#decode/2)
- [euneus:decode\_iodata/1](https://hexdocs.pm/euneus/2.0.0/euneus.html#decode_iodata/1)
- [euneus:decode\_iodata/2](https://hexdocs.pm/euneus/2.0.0/euneus.html#decode_iodata/2)

The `euneus_decoder` options:

```erlang
-type options() :: #{
    codecs => [codec()],
    null => term(),
    binary_to_float => json:from_binary_fun(),
    binary_to_integer => json:from_binary_fun(),
    array_start => json:array_start_fun(),
    array_push => json:array_push_fun(),
    array_finish =>
        ordered
        | reversed
        | json:array_finish_fun(),
    object_start => json:object_start_fun(),
    object_keys =>
        binary
        | copy
        | atom
        | existing_atom
        | json:from_binary_fun(),
    object_push => json:object_push_fun(),
    object_finish =>
        map
        | proplist
        | reversed_proplist
        | json:object_finish_fun()
}.

-type codec() ::
    copy
    | timestamp
    | datetime
    | ipv4
    | ipv6
    | pid
    | port
    | reference
    | codec_callback().

-type codec_callback() :: fun((binary()) -> next | {halt, term()}).

```

Please see the [decoder documentation](https://hexdocs.pm/euneus/2.0.0/euneus_decoder.html) for further information and examples.

### Formatter

Format functions available:

- [euneus:minify/1](https://hexdocs.pm/euneus/2.0.0/euneus.html#minify/1)
- [euneus:format/2](https://hexdocs.pm/euneus/2.0.0/euneus.html#format/2)

The `euneus_formatter` options:

```erlang
-type options() :: #{
    indent_type := tabs | spaces,
    indent_width := non_neg_integer(),
    spaced_values := boolean(),
    crlf := crlf()
}.

-type crlf() :: r | n | rn | none.

```

Please see the [formatter documentation](https://hexdocs.pm/euneus/2.0.0/euneus_formatter.html) for further information and examples.

Please give it a try, and don’t hesitate to join the project on the [Euneus repository](https://github.com/williamthome/euneus) or jump into this thread and share your suggestions. Thank you!

---

<div class="post-metadata">

**Author:** ![williamthome](https://erlangforums.com/user_avatar/erlangforums.com/williamthome/32/471_2.png) [@williamthome](https://erlangforums.com/u/williamthome)\
**Post date:** [August 6, 2024, 9:47pm UTC](https://erlangforums.com/t/euneus-a-json-parser-and-generator-in-pure-erlang/3031/19 "2024-08-06T21:47:41Z")

</div>

# New release

[v2.2.0](https://hex.pm/packages/euneus/2.2.0)

## Highlights

- Decode streaming is now supported: [euneus\_decoder — euneus v2.2.0](https://hexdocs.pm/euneus/2.2.0/euneus_decoder.html#stream_continue/2)
- The function `encode_iodata` is now `encode_to_iodata`
- `codec_callback` added to the encoder options: [euneus\_encoder — euneus v2.4.0](https://hexdocs.pm/euneus/euneus_encoder.html#t:options/0)

Please see the [documentation](https://hexdocs.pm/euneus/2.2.0/readme.html) for more information.

## Changes

- Rename `encode_iodata` to `encode_to_iodata` by [@williamthome](https://github.com/williamthome) in [#79](https://github.com/williamthome/euneus/pull/79)
- Add option to handle unsupported terms encoding by [@williamthome](https://github.com/williamthome) in [#81](https://github.com/williamthome/euneus/pull/81)
- Expose encode functions by [@williamthome](https://github.com/williamthome) in [#82](https://github.com/williamthome/euneus/pull/82)
- Fix typos by [@williamthome](https://github.com/williamthome) in [#83](https://github.com/williamthome/euneus/pull/83)
- Fix doc links in README.md by [@williamthome](https://github.com/williamthome) in [#84](https://github.com/williamthome/euneus/pull/84)
- Add badges in README.md by [@williamthome](https://github.com/williamthome) in [#85](https://github.com/williamthome/euneus/pull/85)
- Implement `continue` encode function by [@williamthome](https://github.com/williamthome) in [#86](https://github.com/williamthome/euneus/pull/86)
- Add option to override the coded callback by [@williamthome](https://github.com/williamthome) in [#87](https://github.com/williamthome/euneus/pull/87)
- Tweak specs for better docs visualization by [@williamthome](https://github.com/williamthome) in [#88](https://github.com/williamthome/euneus/pull/88)
- Remove unnecessary `no_auto_import` flag by [@williamthome](https://github.com/williamthome) in [#89](https://github.com/williamthome/euneus/pull/89)
- Implement decode stream by [@williamthome](https://github.com/williamthome) in [#90](https://github.com/williamthome/euneus/pull/90)
- Add decode stream section in README.md by [@williamthome](https://github.com/williamthome) in [#91](https://github.com/williamthome/euneus/pull/91)
- Tweak stream section in README.md by [@williamthome](https://github.com/williamthome) in [#92](https://github.com/williamthome/euneus/pull/92)

> **[Comparing v2.0.0...v2.2.0 · williamthome/euneus](https://github.com/williamthome/euneus/compare/v2.0.0...v2.2.0)**
>
> An incredibly flexible and performant JSON parser, generator and formatter in pure Erlang. - Comparing v2.0.0...v2.2.0 · williamthome/euneus