Erl_pp:form/1 doesn't print binary strings in -doc attributes

Questions / Help
1

I have a module where the -doc attribute is a binary/utf8 string.

When compiled to abstract form and then back to erlang with erl_pp:form/1 it is printed as an binary integer list and not as a string.
If I use a “simple” string it works fine.

-module(test).
-moduledoc("This is a module documentation").
-export([test0/0, test1/0]).

-doc(<<"Just return the integer 10"/utf8>>).
-spec test0() -> integer().
test0() ->
    10.

-doc("Just return the integer 10").
test1() ->
    10.

Converting forth and back:

2> {ok,[], Term} = compile:file(test,[binary, to_abstr, debug_info]),
    Bin = term_to_binary(Term),
    %% file:write_file("test.abstr", Bin),
    PrettyModule = lists:map(fun(A) -> erl_pp:form(A) end, binary_to_term(Bin)),
    io:format("~ts~n", [PrettyModule]).
-file("test.erl", 1).
-module(test).
-moduledoc("This is a module documentation").
-export([test0/0,test1/0]).
-doc(<<74,117,115,116,32,114,101,116,117,114,110,32,116,
       104,101,32,105,110,116,101,103,101,114,32,49,48>>).
-spec test0() -> integer().
test0() ->
    10.
-doc("Just return the integer 10").
test1() ->
    10.

Is this the expected output or would it be possible to print the binary as a string?

2

It happens because erl_syntax:abstract/1 (which erl_pp uses for this) does not do any printable heuristic for binaries, while the parser keeps that information. For example:

1> erl_syntax:abstract("abc").
{tree,string,{attr,0,[],none},"abc"}
2> erl_syntax:abstract(<<"abc">>).
{tree,binary,
      {attr,0,[],none},
      [{tree,binary_field,
             {attr,0,[],none},
             {binary_field,{tree,integer,{attr,0,[],none},97},[]}},
       {tree,binary_field,
             {attr,0,[],none},
             {binary_field,{tree,integer,{attr,0,[],none},98},[]}},
       {tree,binary_field,
             {attr,0,[],none},
             {binary_field,{tree,integer,{attr,0,[],none},99},[]}}]}
3> erl_parse:parse_exprs(element(2,erl_scan:string(~s'<<"abc">>.', 0))).
{ok,[{bin,0,
          [{bin_element,0,{string,0,"abc"},default,default}]}]}

The best solution to this would probably be to teach erl_syntax to create a binary similar to erl_parse when the string is printable. The code doing this is located here. A PR would be welcome.

3

erl_pp seems to use erl_parse:abstract/1 and not erl_syntax dito, even though this also misses check for printable binaries.
For some reason erl_parse is included in the .gitignore so I cannot get a proper diff, but if I change the abstract/3clause for bitstring to:

abstract(B, A, E) when is_bitstring(B) ->
    L = bitstring_to_list(B),
    case abstract_list(L, [], A, E) of
        {string, _, _} = S ->
            {bin, A, [{bin_element, 0, S, default, default}]};
        _ ->
            {bin, A, [abstract_byte(Byte, A) || Byte <- L]}
    end;

it works fine and I get back:
-doc(<<"Just return the integer 10">>).
UTF8 characters are “lost” (ä → ä) though and I am not sure what happens now with bit strings that are not byte-aligned.