Thank you, though all the credit belongs to @michalmuskala really. 
Nope, just basic encoding and decoding currently, you’ll need to use another package for other features. It could be something to add in future, though I’m unlikely to get to it for some time as I’ve a long list of things I want for Gleam first.
We have now Thoas as standard json library in Nova. You can choose what library you want to use but this was a really good one so we changed the library we used.
Wonderful to hear!
Hey, great work!
Before it is too late, would you consider changing the decode/1 function’s return format to return just the result instead of {ok, ...} | {error, ...}? This way it can be used as a drop-in replacement for any other JSON erlang library, since all the major ones use decode(binary()) -> json() API spec. See jiffy / jsone / jsx
The API is designed to never throw, so instead it returns a result tuple.
It wouldn’t be hard to wrap it in a function which throws if an error is returned. Would adding that function to the user’s codebase be sufficient, or would it be a notable UX improvement to have a thoas_throwing module which exposes that API?
I am in favour of suggestion @seriyps made.
Using tuples makes code less readable, and pushes developers towards bubble-wrapping. Example:
do_steps(Arg) ->
case step1(Arg) of
{ok, Success} ->
case step2(Success) of
{ok, Success2} ->
case step3(Success2) of
{ok, Success3} ->
{ok, Success3};
{error, Failure3}
{error, {step3_failure, Failure3}}
end;
{error, Failure2} ->
{error, {step2_failure, Failure2}}
end;
{error, Failure} ->
{error, {step1_failure, Failure}}
end.
What the developer actually means is:
do_steps(Arg) ->
step3(step2(step1(Arg))).
In most cases “no throw return code” still has to be somehow checked for an error. Often this way:
{ok, Res} = do_steps(MyArg)
Bubble-wrapping has whole lot of associated problems. First, it usually tries to re-create call stack using the Reason in {error, Reason} tuple. Second, it’s cluttering the logic with error wrapping clauses. Third, the actual underlying error tends to be lost in translation (original stack trace is also lost).
Having said that, I would rather have thoas_nothrow returning tuples than thoas_throwing.
Another idea is, if this implementation is faster and 100% compatible with jsone, it makes sense to have a property-based test (jsone vs thoas) proving compatibility. This would let many developers to switch transparently. I would be interested in that.
I have nothing against functions returning {ok, Result} | {error, ..} in general. I’m just trying to highlight the fact that all the other JSON libraries (as well as stdlib’s binary:decode_hex/1 and base64:decode/1) return just the Result from decode/1,2 and throw an exceptions on error.
Developers are used to this API and might be confused that thoas breaks this “unwritten rule”. And also it would be more difficult to swap JSON library in a drop-in like sed -i s/jiffy:/thoas:/-style.
Using tuples makes code less readable, and pushes developers towards bubble-wrapping.
Guess that’s the reason why Eep 0049 - Erlang/OTP was introduced.
Thoas will not default to throwing I’m afraid. I feel strongly that APIs that throw are surprising and error prone, and there’s currently no Erlang static analysis tooling that can assist you in using or refactoring code that throws. Returning values on the other hand is explicit and immediately clear to the user.
Cloning the API of another existing library isn’t a goal so I’m not willing to make what I see as a detrimental change to it in order to meet this non-goal.
Easily solved with sed -i s/jiffy:/thoas_throwing:/, though given all the libraries above accept different options and have slightly different formats for decoded JSON I don’t think the idea of a standardised API is as true as we’re making it out to be here.
Erlang may be getting a solution to this via EEP 49, that would become:
do_steps(Arg) ->
maybe
{ok, Success} ?= step1(Arg),
{ok, Success2} ?= step2(Success),
{ok, Success3} ?= step3(Success2),
{ok, Success3}
end.
That has a different return value, and the errors aren’t exceptional, json errors are very common and part of code flow, shouldn’t be hidden.
And that way will indeed throw if it is not a success, which gives you your throwing transparently while handling errors if you wish as well.
Yep this thing!
+1
It’s trivial enough to make a wrapper module that follows other library API’s but exposes your as well.
+1
Guess could be a compromise.
But then I would expect that thoas:encode/1 also never throws, but only thoas_throwing:encode/1 does. I see that currently thoas:encode/1 has at least one explicit exception thoas/src/thoas_encode.erl at 523c6831aed68b603d427d505d1dd571ddd021da · lpil/thoas · GitHub and I guess a lot of implicit ones (a-la function_clause)
2> thoas:encode(#{"hello" => <<"world">>}).
<<"{\"hello\":\"world\"}">>
3> thoas:encode(#{"привет" => <<"world">>}).
** exception error: bad argument
in function list_to_binary/1
called as list_to_binary([1087,1088,1080,1074,1077,1090])
4> thoas:encode(#{"hello" => make_ref()}). % yes, we are not obeying type specification here
** exception error: no function clause matching thoas_encode:value(#Ref<0.1085671603.2705588226.206231>,
#Fun<thoas_encode.0.100822224>) (/home/seriy/workspace/thoas/src/thoas_encode.erl, line 1717)
It seems entirely fine to throw an exception on bad calls, but bad data is a normal JSON thing to handle and thus is not exceptional?
Implemented this. It works, surprisingly (at least for canonical representation) Add property-based tests (some are failing) by seriyps · Pull Request #12 · lpil/thoas · GitHub
It does not feel idiomatic to Erlang. It is harder to read compared to just step1(Arg). It may be slower and create more garbage due to bubble-wrapping. It loses original stack trace. I would even dare to say it is not beautiful, because code contains useless repetitive characters {ok, Success}.
I am not a supporter of EEP-49 for these specific reasons. It turns Erlang into C by requiring to test every return code, obscuring “happy path”.
Using exceptions allows to handle the error exactly at the level where there is enough context for handling it. In my experience, bubble-wrapped error handling usually looks this way:
handle_call(do_something, _From, State) ->
case do_something() of
{ok, Ret} ->
{reply, Ret, State};
{error, Reason} ->
% swallow all errors and be done with it
?LOG_ERROR("Error, reason ~p", [Reason]),
{noreply, State}
end.
In some cases “error handling” is slightly more elaborate, silently swallowing only selected errors (e.g. malformed JSON) and logging all others.
I find it more powerful to handle errors this way:
try do_something()
catch
error:{can_handle, Context} ->
do_handle(Context);
error:{swallow, Error} ->
?LOG_ERROR("Error: ~p", [Error])
end
This catch does not need to handle all possible exceptions locally. There can be a higher-level try that handles, for example, error:{can_handle_on_higher_level, Info} that had more surrounding context.
Given that pretty much any Erlang code may throw, I prefer to stick to a single idiom and use exceptions for error handling.
I agree with the “static analysis tools” part, tools are not there yet (would be great to fix that).
I disagree with “surprising and error prone”. But I must admit I felt that way before. Could be my personal experience, but having rewritten quite an amount of Erlang code I think I understood exceptions better and fell in love with this concept. It definitely improved conciseness of my code. Made it beautiful.
Yup, this is the idea. If Erlang had a dedicated utf8 string type this could be removed.
Oh very cool, thank you! I’ll review as soon as I have time.
Some suggestions on the API:
Use decode/encode to raise and introduce maybe_encode/maybe_decode for ok/error
Keep decode/encode as is and introduce encode_valid/decode_valid (the idea here is that you expect it to be valid and therefore it should raise if not)
Use a separate module but it slightly feels like overkill
I see jsone library exports try_encode / try_decode functions for {ok, ..} | {error, ..} and encode / decode which return plain results and throw on error.
Oh… Well, that makes sense I guess.
v0.3.0 is out, now with support for encoding proplists
Thanks to Michael Davis and Michael Klishin for this
Any plans to support (existing) atoms as keys?
Nothing is planned for this presently.
Is there some technical reason or would you be open to adding it (or a PR)?