I’ve just submitted a pull request about adding new type of function exports. This thread is where it all started, and this is just one of the use cases.
TLDR: internal_export is an attribute that gives you more control over function exporting. In some cases, functions are only exported for internal usage, for testing purposes or because you must implement behavior callbacks. Those functions are not meant to be used by the end user of your application and they shouldn’t exists in public interface of a module. Using them is currently considered bad practice, but with this feature it can be integrated into testing pipelines.
Best description of this problem I’ve found is from dr. sc. Richard A. O’Keefe: “there are many “applications” in the Erlang/OTP system which consist of a flotilla of modules. A module in such a flotilla cannot export it to another member or other members of the flotilla without exporting it to absolutely every module”
PR message contains all the details, so take a look. Any feedback is welcome
thank you for implementing this!
IMHO this has real potential for use in tooling and in general to resolve class of code-organization and documentation issues while not interfering with elegance of erlang (66% is fairly large number in this context).
I hope this receives positive feedback from the community and gets into future release.
I recently ran into a similar issue, but with types, not functions: there’s a fairly complex type in my application that multiple modules use, but it should be opaque outside the application. So I do think this is a good idea and could be even extended to types.
Kinda agree, new stuff must be added with care to keep language simple enough, but also new stuff should be added sometimes. There isn’t extreme necessity, but stated problems do exist. Whole solution is only 110 lines long, so I don’t think it is a complex feature that would produce noticeable maintenance costs on a long run. I think this feature would helps formalizing “don’t use it if it is undocumented” practice (especially in combo with Erlang LS and new shell autocomplete).
Not really sure how is it related to Elixir (or is it that every feature “elixirizes” Erlang).
Exactly, there is a symmetric problem with type exporting, I think @kennethL proposed it in some thread. I’m actually already working on it, but just wanted to get some feedback on internal_export.
I did read the PR, but I’m not sure about how restrictive this feature is meant to be. Like, if it should raise just a warning when a module uses an internal_export function, not include the info in the module_info and in the shell autocompletion etc; or if it is supposed to refuse to compile altogether.
The former, yes, count me in The latter, no, I wouldn’t like that
That said, I’d opt for application-level restrictions. Restricting it to modules would be too much hassle. Plus, application-level restriction would happen pretty much automatically, while the modules that are allowed to use an internal function would have to be denoted in the module itself, which therefore would need to know about modules it would be used by.
And that said, I would encourage creating the ability of doing the same for entire modules. Some are meant to be only used for the application that they are part of, not outside, helpers, that kinda stuff.
and as far as i know this proposal is implementing application-level restrictions. PR is also static check only for now with possibility to implement dynamic check later on if there is interest.
This PR does the former one - xref analysis. You can use it to get warnings, but you don’t have to, it’s up to you. In EEP05 the later was proposed, to embed the mechanism in the code loader, but IMO that would only be reasonable in combination with dynamic checks.
You can do it for entire modules - just mark all exports with internal_export There could be some attribute like internal_module or something, but I think there’s no need for it.
Correct. TLDR: You can statically call internally exported functions only from within your application. (Dependencies of your application can also call them statically, but it is anti-pattern - that would mean that your dependency is forcing you to name a module by their convention.)
I would love to hear how this is an attempt to “elixirize” Erlang when:
Elixir does not have this feature
The proposal does not mention Elixir
The EEP 05 which first proposes this feature predates the creation of Elixir by at least 4 years
If you do not agree with the feature, could you be more precise with the feedback? Or is Elixir such a bad-word that we can just throw it around to say a feature is bad without any further explanation?
It is, but this only solves “test-only” exports. Goal of the feature is not to solve that problem, it is already solved. The goal is to clarify module interfaces and interfaces between applications, but due to the nature of the solution, it can also be used for “test-only” exports.
Are you saying “test-only” exports is a solved problem? My staff was just asking me about this issue. Outside of this proposal, how are “test-only” exports currently addressed in Erlang? Thanks!
But, as usual, a warning: If you find yourself doing that, there is a very good chance that your tests are not reflecting the actual usage of the system. It would be a good idea to verify if they can’t be improved to remove the need for non-exported functions.
I think it’s a fine feature, provided it only affects xref or dialyzer checks and doesn’t restrict ability to call internal functions in the runtime. Sometimes we have to call arbitrary functions from the erlang shell or via nodetool during troubleshooting sessions in live or test environments, and this is a very important aspect.
OK I reviewed the text of your pull request and Eep 0005 - Erlang/OTP . I think your arguments are compelling but, what’s even more compelling is working code. Glad that exists. What’s not clear is what happens when, at the erl repl, I start calling functions in a module declared as internal_export. Does it return an error, a warning, or silently let me get away with it?
The most obvious use case for us is for being able to create unit tests for “internally public” functions but can definitely see other ones that, should this feature be working, we might avail ourselves of.
REPL is runtime and this feature is only part of the static analysis tool, therefore nothing will happen - no errors or warnings.
However, it could be nicely integrated with the new shell (OTP >= 26.0) to separate exports and internal_exports in autocomplete to provide better dev experience.
In this case, we have decided to reject EEP 67 on the basis that documentation attributes, introduced in OTP-27, is expressive enough to cover internal exports. Anything that has the documentation attribute -doc false. or -doc hidden. does not produce the documentation and can be understood as an internal function.
One can throw warnings using xref or using Dialyzer, by adding a new option that throws a warning when a hidden module is used externally.
However i must express my worry about decision making process in general and specifically surrounding documentation features that popped up here as well.
How can it be reasonable that we are supposed to express non documentation details with documentation parameters?
Sure there was practice in past to say “what is not documented should not be used” but is this really how we should think about this?
Internally exported is attribute regardless of documentation and if anything documentation should respect this attribute instead of expecting user to understand all the details and intricacies of documentation.
why are we starting to make decisions that are logically backward?
let documentation attributes dictate visibility of functions instead visibility of functions dictating documentation profile
introduce tool that is producing worse documentation and then hope it will be fixed instead of fixing the tool first and then introducing it (which i have alreade expressed here )
I’m calling once more for the OTP team to consider holistic approach of doing things and maybe step on the breaks a bit, because this has real potential of damaging erlang as a language especially for the new users (assuming we want new users).
Conversation with new user:
N: i want to export this for callback but i don’t want user to use it?
You: oh it’s easy just put -doc false or -doc hidden attribute
N: say what?
N: are you kidding me?
N: How was i supposed to search for that?
N: But i’n not currently even writing documentation
N: xref should be aware of documentation?
N: how many of these things are hidden in the language/ecosystem ?? (that i can not easily find)
