Comments on merge request OTP/#8026 - Change documentation to use ExDoc

[Maria-12648430]

TBH, I was more or less just thinking out loud there, without giving it much thought

[kuna.prime]

Just to simplify this discussion here is an example of spec dumping:

to be clear this is not complete spec dump but only part that i was able to capture on single screen.

Please also notice “semantics” of since attribute.

[robert]

Thank you for the clarification, everyone. I misunderstood the entire thing and I apologize if my intention comes across as me wanting to spread misinformation, far from it.

I guess in simple terms I wish that changes that affect the users of the language could be discussed more in the forums, and feedback collected from users (irrespective if said changes are discussed, voted on, etc. in other circles.) Since the forums (and before, the mailing list) are the one point that is common to most users, it follows that it should, perhaps, play a more central role in discussing changes.

Some changes are discussed, others come as a surprise. Reading this thread makes it obvious that I am not the only one confused by this, though.

@peerst
@MononcQc

[garazdawi]

@kuna.prime

As to if this is only temporary visual regression … ? I’m not so sure about. There are issues that are really not only design/HTML/CSS.
For example mayor regression is “spec dumping” and given that even official Elixir documentation is doing that (and no one seem to be bothered by it) are we realistic to believe that will change?

If enough people agree that it is a major regression then it will be fixed, if not, then it will not. Its the same with the current documentation infrastructure. Yes I agree that functions such as erlang:system_flag/1, erlang:system_info/1 and similar are pain points that need to be solved. I’m not sure the old approach was any better though as we had to resort to manually writing overview sections for a user to be able to read it. Since dialyzer cannot use any (or very little anyway) information in those types anyways, I may collapse it to -spec erlang:system_flag(Flag :: atom(), Value :: term()) -> OldValue :: term() and describe the values in text. Or do something else.

@kuna.prime

If erlang didn’t already have amazing documentation this would not be the issue but the fact that it does makes this big regression.

I find it a bit funny when seeing this and then comparing to what the people used reading docs in ExDoc say. Things such as “This is such a massive step in the right direction. The old Erlang docs felt like a relic from the 90’s” and “I didn’t realize just how amazing this would be. WOW”. Familiarity with a tool does a lot to make you like it.

I like our docs the way they are, having contributed a lot to how they look myself. But ExDoc does a lot of things right, like exposing User’s Guides and Modules to the user directly instead of hiding them behind a link, hovering over links give the summary of the target, better responsiveness, easier to group various related things (modules, pages, function, types) and more. These are all things that we could do in erl_docgen, but we haven’t and probably wont if we remain with it.

ExDoc is also historically tailored towards how Elixir code looks and works. This is what makes it not as good for some things we like to do in Erlang code. They have had 10 years to make it work great for them, we’ve had 3-4 months.

@kuna.prime

Please also notice “semantics” of since attribute.

This is an effect of the tool used to do the convertion. Any function with multiple sinces will need to be manually fixed. See how it is done for erlang:system_info/1 for a better example. The spec is still there and not helping a lot, but the text is more like what the end result will look like.

[garazdawi]

@robert

Since the forums (and before, the mailing list) are the one point that is common to most users, it follows that it should, perhaps, play a more central role in discussing changes.

We try to have discussions regarding changes we make to the language and tools in the forums. It will always be a tradeoff about what we discuss here and what we don’t. Sometimes we post things and no-one is interested at all, and sometimes the opposite happens. I did not expect there to be such an interest in the choice of tool to create the Erlang/OTP documentation.

The discussion that was a precursor to this change is:

Implementation of EEP-59 - Documentation Attributes in Erlang

Implementation of EEP-59 - Documentation Attributes Documentation attributes are added to the binary beam file, following format of EEP-48, via +beam_docs compiler flag Warnings related to documentation attributes are dealt with in the beam_docs.erl instead of adding them to erl_lint.erl Example 1 -module(warn_missing_doc). -export([test/0, test/1, test/2]). -export_type([test/0, test/1]). -type test() :: ok. -type test(N) :: N. -callback test() -> ok. -include("warn_missing_doc.hr…

While not stated explicitly, I think that if one looks closer into the PR it becomes apparent that we intend to change the way we generate the HTML docs.

[kuna.prime]

@garazdawi

They have had 10 years to make it work great for them

this only fortifies my point as they made something that is objectively worse than what erlang has.

Don’t get me wrong, there are many things that current documentation can do better but the core thing that documentation does: presenting information of the system/API in clear and readable way to the user is superior in erlang world than in elixir world.

[robert]

While related, the changes discussed there are not exactly the same as the ones discussed here, unless I misunderstand it. I have followed that discussion at the time, I had no objections.

In any event, I hear you.

[peerst]

No offence taken. Let me offer more explanation:

EEF doesn’t have a direct say in the future of OTP, thats done by the OTP team and they already have the EEP process which is open to the whole community and the discussions on EEPs are taking place here on this forum (previously on the mailing list).

Occasionally EEF gives stipends to projects that might end up as a PR to OTP (after consulting with the OTP team if they are interested) or most of the time to maintain or create new open source offerings. The technical discussions about what EEF spends money on happens in working groups of volunteers and the only requirement to participate is the free membership tier. Otherwise we try to grow the community and provide infrastructure. One of these are the EEF Slack where many WG organise their meetings, having such a members only Slack makes sense as an organisation and it gives us the possibility to pay for the account and retain history (currently a free trial expired, so we are working on making that happen) which we couldn’t afford for Erlanger and Elixir Slack.

Its best to come and see: Join Us - Erlang Ecosystem Foundation

BTW the whole organisational structure and membership tiers for EEF was leaning heavily on the Python foundation whose community can be considered successful I think. We are a 501(c)(3) non profit and required by law to be transparent so really no secrecy here.

[carlotm]

@kuna.prime

Don’t get me wrong, there are many things that current documentation can do better but the core thing that documentation does: presenting information of the system/API in clear and readable way to the user is superior in erlang world than in elixir world.

I couldn’t agree more (I am doing Elixir daily).
The only thing I am missing on Erlang docs are examples.
But the presentation is amazing.

My 2c

[josevalim]

@kuna.prime

this only fortifies my point as they made something that is objectively worse than what erlang has.

Don’t get me wrong, there are many things that current documentation can do better but the core thing that documentation does: presenting information of the system/API in clear and readable way to the user is superior in erlang world than in elixir world.

That’s the wrong conclusion.

We have made a system that renders well the docs as Elixir have historically stored them, while Erlang docs were written for another system. Now that Erlang docs are using the same tool, it is 100% expected to find rough edges. You said perfectly, some things will be better, some will be worse. But it doesn’t mean we can’t improve what is there and we are very open to issues and pull requests to continue improving it. It is an evolving system and a big part of the premise is that, if we all focus on evolving it together, then we all stand to gain from each other’s work.

If the contrast is worse? Let’s open up an issue and discuss that too. ExDoc maintainers are also frequent Erlang users and we also want something that is perfect for us.

[josevalim]

@juhlig

After a while, I will only want to know what a described function does and how to use it, I won’t care how nice and colorful it is presented to me. In that respect, the current/old style is pretty much perfect, the new isn’t, it puts too much emphasis on looks at the expense of readability.

We don’t really know if that’s factually true though. For example, I have used both Erlang and Elixir docs extensively throughout the last decade, I am extremely used to ExDoc, and I’m still adapting to read Erlang docs in ExDoc, which makes me think a lot of it boils down mostly to habit.

I can factually say that:

• We have double checked all of ExDoc to conform Web Content Accessibility Guidelines, at least level AA and we would be glad to push to level AAA (if not there already). I will do another pass later today.

• We had several contributions focused exclusively on improving the accessibility of the docs for those using screen readers

• ExDoc provides a dark mode with high contrast (which was highly requested)

Other than that, it boils down to opinions, and I am not sure that they will lead anywhere useful. That said, I 100% agree with @Maria-12648430’s suggestion, we could hire a specialist on accessibility to actually tell us what to improve, and I will make sure all of the suggestions are properly implemented.

[josevalim]

@nzok

The red text screams out in a way that the blue text does not. This is contrast going the other way. But the red text is not the most salient part of the information, so it should not be the first thing that catches the eye.

The documentation system derives several colors from the main color, which turns out to be red in Erlang’s cases (quite similar to this forum). This means a derived color may be used somewhere which may be undesired.

The good news is that ExDoc is open source, so we can tweak it, and the CSS was mostly written by competent front-end people (aka not me :D), so changing ExDoc to use separate styles for Erlang and Elixir is quite straightforward because most font and colors are compartmentalized into few CSS variables. I have submitted a pull request here:

Use blue links for Erlang

Everyone is welcome to give a thumbs up/down stating your preference in the pull request above (or drop a comment).

I hope this motivates folks to give it a try and help improve areas you find lacking. Not all issues are going to be this easy, but many of the concerns related to readability can be addressed with a few lines of code.

@kuna.prime

Just to simplify this discussion here is an example of spec dumping:

“Since” is likely a bug on the Erlang generation side. Regarding spec-dump, this is tricky. One of the downsides of the current implementation in Erlang is that it is impossible to easily see all supported flags. We don’t have this problem anymore but now there seems to be too much information. I wonder if we could perhaps improve this by improving the display? We could use indentation and newlines to give more spacing, it will take even more vertical space, but it should make it more digestible. Thoughts?

[nzok]

Two problems with voting.

1. It took a lot of searching to find the voting buttons. Yellow against white really doesn’t work.

2. In order to vote at all, you have to sign in to GitHub. This is certainly going to create a biased sample. As for me, I do have a github account, but github sends the confirmation message with the code I need to complete signing in to an old e-mail account, and I can’t change that without signing in… I have asked for the old account to be temporarily reinstated so that I can change it, but in the mean time, I am unable to vote.

3. The blue links are SO much better that it really isn’t funny.

4. A key point in information presentation is that the most VISUALLY distinctive information should be the semantically most SALIENT. What is easiest to see, even from a distance, should be what you most need to know. Links need to be distinguishable from non-links, IF anyone wants to look for them. Most of the time, the links are not what you most need to know. This is not a issue of taste. This is not an issue of aesthetics. It is an issue of communication. Looking at any chunk of text, if the reader can only read one word / sentence / paragraph, which one is it? THAT is the word / sentence / paragraph that needs to stand out by being larger or a heavier typeface or bright colour or a box or something.

[josevalim]

I have just created something quick and informal. You’re also welcome to set up your voting platform and methodology of choice and let us know of the result. The whole point is that we are open to feedback, contributions, accessibility reviews, etc.

Overall, I agree that blue is better and respected user experience institutes, such as Nielsen, say as much. So I could have pushed the change directly, but I thought it is a good opportunity to get folks involved. It also seems to be everyone’s preference anyway.

[AstonJ]

Why not use the Erlang colours and slightly increase the size of the mono font (to 1.1em from .9). I think it is more in-keeping with the Erlang branding:

[kuna.prime]

@josevalim

Now that Erlang docs are using the same tool, it is 100% expected to find rough edges.

if my observations are “rough edges” i would not even start this thread as i would consider that to be natural part of progress in any project/change.

i’m also very glad that real rough edges are starting to be discussed and addressed (colors, fonts, contrasts and links).

also i must admit that my initial post was lacking due my assumptions how conversation might go, and i’m also known to be not so good at properly communicating things.

however during this thread and also by examining root causes of my complaints (everything already expressed in my posts here so i don’t want to bother people by rehashing those again) i become increasingly convinced that ExDoc is it it’s current state is the wrong tool for the job, and that no amount of “rough edges” discussions and patches will fix root causes.
main problem is that ExDoc is automating rendering of function interfaces by “spec dumping” when it is not even clear that this kind of automation is helping at all. there is possibility that this automation is toxic (forcing inferior documentation) and that sometimes ( frequency is currently hard for me to ascertain ) manual approach is superior to the point it is not even funny.

@josevalim

Regarding spec-dump, this is tricky.

i agree and it is not in any form or shape related to any kind of implementation in erlang. on contrary this is introduced by inconsiderate practice by ExDoc that seems to be reasonable at the first glance but in practice it is absolute mess.
we can live with degraded experience for sure, we can hope for fix, but maybe we should first fix the tool and then use it?

and by the way, i completely understand, respect and support OTP’s team enthusiasm for convenience of the tool, but i’m very worried about the product (this time product being documentation).

[josevalim]

@kuna.prime

i agree and it is not in any form or shape related to any kind of implementation in erlang. on contrary this is introduced by inconsiderate practice by ExDoc that seems to be reasonable at the first glance but in practice it is absolute mess.

You are incorrect on the root cause. In Erlang, you cannot associate a given specification with a given function clause (and to a given documentation). This is not a feature you can express in the source code.

You can even see this by exploring how the feature you request is implemented in “old” Erlang docs. Each “clause” in the docs:

erlang/otp/blob/ace566d1317b26b107fa332774b5b2b0cc030e56/erts/doc/src/erlang.xml#L6574-L6575

has to point out to an index (note the clause_i attribute) of the spec dump in separate source code file:

erlang/otp/blob/ace566d1317b26b107fa332774b5b2b0cc030e56/erts/preloaded/src/erlang.erl#L2667-L2669

which is extremely brittle and hard to maintain.

---

Of course, we could implement the same system in ExDoc if desired, but without fundamentally changing Erlang (all the way to the parser), the best we can do is to have documentation chunks pointing to indexes in the existing source spec dump, which is messy and error prone.

Maybe you still believe the ends justify the means here but my point is that the trade-offs are more complex than they seem to be and the root cause also runs deeper than documentation tooling.

I’d also repeat that the old documentation makes it harder to understand everything a function does, because the definition of process_flag, system_flag, etc are scattered across several pages, and the “spec dump” solves this problem.

I am 100% fine with making ExDoc behave either way, on the documentation generation side it is not an issue, but I completely understand the Erlang/OTP team if they believe the trade-offs are not worth it and they decide to put the work on making the spec dump more accessible. We could also try to have both: have the specs dump at the top and, divide the docs for said functions into several sections, manually duplicating the spec clause inside each individual section. Picking a solution will require navigating these trade-offs, instead of dumping all the blame on a single place.