Open sourcing erlfuzz

Open sourcing erlfuzz

What is it?

erlfuzz is a new tool for generating random Erlang programs. It is designed to test various pieces of the Erlang ecosystem such as the erlc compiler, the BEAM VM, eqWAlizer, dialyzer, infer for Erlang, erlfmt, etc…

It has been under development since last summer, and has just been open sourced: GitHub - WhatsApp/erlfuzz: erlfuzz is a small standalone generator of random erlang programs used to fuzz the erlang compiler and VM, as well as other tools such as erlfmt, dialyzer, eqWAlizer, etc... It does not currently use coverage information or do anything especially clever, except for carefully handling Erlang's scoping rules.

Strengths and limitations

Its main features are the following:

  • Support for most of the language, including some new features such as maybe expressions and map comprehensions.

  • A very fine-grained understanding of Erlang’s scoping rules, allowing it to generate complex code that reuses variables without generating invalid Erlang code.

  • An automated test case minimizer, that usually reduces the test cases from several thousand lines to a couple of lines.

It currently has some limitations to be kept in mind:

  • It does not use any coverage information
  • It generates quite a few ill-typed programs
  • The test case minimizer only works on test cases found by the fuzzer


Erlfuzz has found so far:

  • 62 bugs in erlc
  • 7 bugs in the VM
  • 6 bugs in dialyzer
  • 5 bugs in eqwalizer
  • 2 bugs in infer
  • 1 bug in erlfmt

While most of these are crashes of the tools on rare code patterns, several bugs are more interesting. For example:

erlfuzz has also been successfully used to detect bugs in PRs before they were merged. The most extreme example is Lift restrictions for matching of binaries and maps by bjorng · Pull Request #6415 · erlang/otp · GitHub (which itself was motivated by corner cases of the language discovered by erlfuzz). It went through multiple rounds of revisions as the fuzzer found 8 different bugs in it over time.

A more surprising benefit of this work was to improve the official Erlang documentation. I tried to make Erlfuzz able to generate any string that corresponds to a valid Erlang program according to the documentation, and this revealed several places where the documentation was either ambiguous, or outright missing some rules. E.g. Address a few omissions in documentation of comprehensions by bjorng · Pull Request #6929 · erlang/otp · GitHub.

Finally, Björn Gustavsson made a creative use of the fuzzer, adding crashes on lines of the compiler that were not covered by the test suite, and getting minimized test cases to add to the suite for many of these lines, improving the test coverage: Cover more code in the compiler application by bjorng · Pull Request #6872 · erlang/otp · GitHub.

Future work

The main priority at this point is to integrate erlfuzz into CI for all of the tools mentioned above.

I hope to cover the last few missing parts of the language later on, as well as try to generate more well-typed programs to hit code deeper in tools like eqWAlizer.


I owe many thanks to Björn Gustavsson and John Hogberg from Ericsson for fixing with stunning speed all of the erlc and BEAM bugs that I reported.

I’m also grateful to my colleagues at WhatsApp that helped me with various aspects of this work, from getting eqWAlizer and infer working to reviewing all of my patches.


I think erlfuzz is a wonderful piece of work and I am impressed that it was
developed, happy that it was used, and grateful it has been released.


Very well said. I’m doubling down on the being impressed, happy, and grateful. And kudos to the OTP team on addressing those quickly (as usual!).


There is no God

I did wonder if someone had built themselves a fuzzer.

1 Like

What is the difference between this tool and PropER? In my head, what is described above was what PropER already did. Granted, I’ve never used either one.

1 Like

This is a good question. PropER offers tools to generate inputs to Erlang code, so it could certainly be used to test erlc as well. But the benefit of building a more specific tool like erlfuzz (that only generates Erlang code, and not arbitrary data) is the possibility to customize it greatly.
For example, erlfuzz has a very precise understanding of Erlang scoping rules, knowing things like which language constructs allow shadowing, or which variables defined inside a case expression can be used after it. I do not believe this would be achievable with PropER.
erlfuzz can also be used to fuzz non-Erlang tools as long as they ingest Erlang code, such as eqWAlizer (which is primarily in Scala with some Rust).