New BEAM There, Done That with Tobias Pfeiffer covering Benchee and the practical details of measuring BEAM performance correctly.
The Erlang-relevant discussion:
Reductions. Tobias learned about them from a GitHub issue and had to look up what they were. A reduction roughly corresponds to a function call -it’s a unit that increments with work done, independent of wall-clock time. Benchee measures it by monitoring a process, recording the reduction count at start and end of each benchmark function call. The result is deterministic within a given OTP version and machine-independent.
Memory measurement. Same approach: spawn a process, monitor GC events, report exact bytes used. For deterministic code, memory is always the same. Both measurements can vary across OTP versions -Benchee reports the full runtime environment alongside every result for this reason.
The tail recursion lesson. Body-recursive map outperforms tail-recursive map for small lists on the BEAM. The “always tail-recurse” advice is correct at large scale but misleading as a blanket rule. Input testing across sizes reveals this immediately.
Design: Benchee has no macros and no DSL, deliberately. All functions, fully composable. The community has built CSV, HTML, markdown, and LiveView formatters on top of it because the underlying interface is stable and public.
