Rebar3_ex_doc overview / usage documention feedback

Off subject from this post, but I am interested in how we can improve the README to provide a smoother entry for those starting off with rebar3_ex_doc and ex_doc in general.

rebar3_ex_doc v0.2.16 — Documentation for reference.

Edit: For context this started off as a few replies in an erlperf thread, but felt deserving of its own thread. There seems to be some rough edges as far getting up and running with rebar3_ex_doc and I’m interested in gathering feedback on how to improve this area.

2 Likes

I learn about it in the elixir doc
generates a single documentation for all child apps
I try umbrella in erlang, but it will make for each one, I think this is useful :grinning:

2 Likes

Yeah, I think an issue is it’d be nice to provide a lot of documentation about ex doc, but it’s also doubling up effort, potential for dry rot, etc. It’d essentially be a lot of copy / paste in other words, thus why we link to the docs for ex_doc.

One thing that may help possibly is having some example projects in the repo, possibly?

2 Likes

In case it’s helpful to someone, this is the relevant part of my rebar.config:

{project_plugins, [rebar3_ex_doc]}.
{ex_doc, [
    {extras, [
        {"README.md", #{title => "Overview"}},
        {"LICENSE.md", #{title => "License"}}
    ]},
    {main, "README.md"}
]}.

Potential gotchas I’ve come across:

  • In the module code, as far as I can tell it only supports the minimal subset of tags supported by both edoc and exdoc. I stick to @doc [... @end].
  • It seems to be impossible to associate a comment with a -type definition, which is annoying since there is a section created for types.
  • Type specs are formatted inconsistently depending on how long they are (i.e. shorter ones are on one line, longer ones break lines). This is in contrast to the Erlang documentation.
3 Likes

It is possible, see example. It just feels unusual, as you need to put documentation after the actual type definition.

The real issue is that all types are reflected in the documentation, including internal types that aren’t exported. I didn’t try to troubleshoot whether it’s EDoc or ExDoc issue, but it should be rather trivial to fix in either of the tool.

As for the tags supported by ExDoc, it knows about {@link tag and also automatically linkifies a module name, or mod:fun/arity in the code block, e.g.

`erlperf_cli' 
becomes a link.
3 Likes

TIL :slight_smile:

It would actually be quite nice to be able to expand the documentation of a record type, showing the types of its fields. However, that seems to be a deliberate design choice by edoc since according to the manual:

Tags are associated with the nearest following program construct “of significance” (the module name declaration and function definitions). Other constructs are ignored; e.g., in:

%% @doc Prints the value X.

   -record(foo, {x, y, z}).

   print(X) -> ...

the @doc tag is associated with the function print/1.

Another question: I copied the format of the source_url/source_ref options in the example rebar.config. It works fine, but I get the following warning:

# rebar3 ex_doc
===> Verifying dependencies...
===> Analyzing applications...
===> Compiling aoc2022
===> Running edoc for aoc2022
===> unknown ex_doc option {source_ref,<<"main">>}
===> Running ex_doc for aoc2022
2 Likes

Oh, forgive my unclear expression
I just want to say if erlang can support generates a single documentation for all child apps
I have try ex_doc in elixir and it’s good enough for me

my test step:

rebar3 new release umbrella
cd umbrella/apps
rebar3 new app app1
rebar3 new app app2

modify every rebar.config

{project_plugins, [rebar3_ex_doc]}.
{ex_doc, [
     {extras, ["README.md", "LICENSE"]},
     {main, "README.md"}
]}.

I expect it will create umbrella/doc just like elixir, then I don’t need to write something to link all apps

2 Likes

It is not “unusual”, it is the way EDoc has always worked. It is not an Rebar / ex_doc issue.

There are perhaps a few bugs or exceptions in the Rebar plug-in, but in general if your documentation looks proper with normal EDoc it will also look okay in ex_doc.

3 Likes

No question it is due to the way EDoc has always worked. However it is unusual from a user experience, because it is inconsistent with the way that developers typically comment code, and with the placement of explicit @doc tags. My preference would be that EDoc had a broader minded view of what it considers a program construct “of significance”.

2 Likes

I agree, I just wanted to point out that it is not “unusual” from an EDoc users perspective. :stuck_out_tongue:

However I do think module documentation could be greatly improved in Erlang (see for example Eep 0059 - Erlang/OTP)

5 Likes

Since I’m the last person ( hopefully not the last one alive :wink: ), who did some significant work on EDoc, I think I can share some of my impressions.

Exactly. Generally speaking, everything that existed in the language when EDoc was created has good, though sometimes messy/redundant, support. Everything that got added after it has been created had poor/hacky or nonexistent support. This meant:

  • behaviours / callbacks
  • types and specs
  • possibly also records

It’s evident upon inspecting the code, no doubts about that. Implementing EEP-48 changed some of that, but to make it happen it had to significantly modify some core parts, too, yet keep the erl_docgen integration with the OTP build system intact, which was a major pain in the neck. Trailing comments on types are an example of this poor & hacky support.

There were some discussions going on before [1], during [2], and after [3] the EEP-48 implementation happened, as well as some next steps [4] listed. I don’t think all of these “next steps” have been addressed until today (even though the issue is now closed), but I don’t think that effort should be put on the shoulders of a single unpaid volunteer either (be it me or anyone else).

The TL;DR of the above links is that:

  • types can have trailing comments and since Erlang 24 callbacks can have trailing comments, too:

    -callback f() -> ok.
    %% This is a callback doc comment.
    
  • however, due to the reasons described in [3] comments on callbacks and types are limited to plain text and cannot have any tags, so no @since, no @deprecated, no @see, etc

  • if we use rebar3 ex_doc then doc chunks are put under _build/docs/lib/<the-app>/doc/chunks, so to access them in the Erlang shell we have to run the shell via rebar3 as docs shell; otherwise we might be surprised that there are no docs in the shell, though there are doc comments in the source

  • it’s possible to generate ExDoc docs for some OTP libs; with some effort it would be possible to upload them to https://hexdocs.pm

  • some formatting options like plain text lists don’t get rendered by EDoc as we would, having got used to Markdown, expect them to be; this means we have to resort to HTML tags

  • but only the set of HTML tags supported by the shell_docs renderer is supported in the chunks; this means, for example, no tables

  • it’s possible to link to various entities using the EDoc link syntax: {@link //app/mod:fun/arity}

  • there are some EDoc tags which could be removed:

    • @spec and @type which since Erlang 24 are deprecated and superseded by Dialyzer/Gradualizer -spec and -type attributes; EDoc already supports the attributes and prioritises them over the old-style tags if both are present
    • @deprecated which seems to be better handled by Xref -deprecated attribute; EDoc could also recognise this attribute and report accordingly; on the other hand, if @deprecated could be used also on types or callbacks it would be a use case not possible with Xref -deprecated :expressionless:

There’s a recent PR to ExDoc to fix that, but I believe it should rather be addressed upstream in EDoc. I’ve volunteered to prepare a patch, but anyone should feel free to beat me to it :wink:


Generally, I think there’s still quite a bit to improve. But how can we do it? By chipping in!

If you’re an individual, then scratch your own itch. Pick your biggest EDoc / rebar3_ex_doc pain point and send a PR fixing it.

If you represent a company, sketch a grant/bounty plan or at least signal interest and I think something will happen under the EEF umbrella if there’s sufficient interest.

I’m happy to mentor newcomers willing to work on EDoc or get my hands dirty again if there’s a wish for bigger documentation improvements.

6 Likes

To be complete, you do not have to represent a company. If you want to work on this and have a good idea of a goal and possible path, I think the EEF could totally be interested in funding a grant for your time.

Put otherwise, the volunteer does not have to do it for free even if during their “free” time.

If you have no idea how to go through that process, feel free to contact me. I have shepherded my own proposal a couple of times through this process and I would be happy to help you. While the EEF does not pay amazingly well, they pay. And if you were going to put in the time anyway, better get something for it.

5 Likes