We’re working on Rebar v4, and one of the things we need to do is make sure the changes we’re making continue to work well with the plugins people actually use in real projects.
@bchassoul is leading this part of the work, and we’ve put together a first ranking of Rebar plugins that we think would be most useful to include in compatibility testing.
How we ranked the plugins
Rather than trying to test every plugin, we ranked them using five criteria:
- 25% – How widely the plugin is used
- 25% – How much of Rebar it exercises
- 20% – Whether it is maintained and current
- 20% – How useful the workflow is to cover
- 10% – Whether it can run reliably in CI
We started with the recommended plugins listed on the Rebar3 website, but didn’t use that list on its own. A plugin can be listed there and still be a poor fit for regular testing. We also considered community usage, which parts of Rebar the plugin relies on and whether we can test it reliably.
Our current top 10
rebar3_hex– 93: Package build and publishing workflowspc– 91: Native-code builds, hooks and clean behaviourrebar3_ex_doc– 87: Documentation, project metadata and dependenciesrebar3_lint– 84: A familiar CI and provider workflowcovertool– 82: Test output and coverage reportsrebar3_gpb_plugin– 80: Code generation and pre-compile hooksrebar3_format– 74: Formatting in CIrebar3_proper– 73: Property-testing workflowsrebar3_appup_plugin– 70: Release upgrades and appup generationrebar3_path_deps– 68: Local and path-based dependencies
We have another 10 plugins in the ranking, with priorities ranging from nightly or targeted testing to lower-priority legacy smoke tests.
A couple of things we decided along the way: aliases belong in Rebar’s own tests because alias support is built in, and rebar3_vendor has been superseded by rebar3_path_deps, so we don’t think testing both would add much.
We’d like your input
Does this top 10 reflect the plugins and workflows you actually depend on?
Are we missing a plugin that should have a much higher priority?
When thinking about it, we’re especially interested in the workflows you rely on: publishing packages, compiling native code, generating code, testing, making releases… or something else entirely.
We’re not trying to defend the scores. The goal is to build a test suite that catches the kinds of regressions that would actually get in your way.
@bchassoul has written up the full ranking, including all 20 plugins and how we arrived at it, here:
Which Rebar Plugins Should We Test First? 
→ Which Rebar Plugins Should We Test First? 👀 | GRiSP
Thanks in advance for any feedback, especially from people maintaining or regularly using these plugins.
And thank you to everyone supporting the Rebar v4 Kickstarter work. Backers will also receive a separate poll so they can weigh in directly.
