Hi! After years of Erlang/OTP I switched back to Compilers/LLVM, and I can’t stop thinking about both topics as potentially compatible and making a profit from each other.
I wonder if it is feasible to bring LLVM optimizations and a backend to Erlang? Were there any attempts and estimations recently? In other words, is it an open question, or rather it’s been thoroughly thought about and thrown away for the end of days as making no sense?
I read of course all that I’ve been able to find about the subject, but that’s not much. Maybe someone has insights to share here?
– Vyacheslav
1 Like
Hello!
HiPE had an LLVM backend before it was removed in favor of the current JIT. There is a paper called “ErLLVM: an LLVM backend for Erlang” that explains the pros and cons of using LLVM.
My reply about using LLVM for the JIT in the original JIT PR still stands:
LLVM is still too slow for what we want to do here. We used llvm in other JIT attempts, but we always had issues with the time it takes for the compiler to run. The sea of nodes approach that llvm ir uses just can’t be as fast as we need it to be, even if you disable all of the optimizations. Maybe if you emit machine instructions instead of llvm ir, but even then I’m doubtful.
Also, as you mention the size of the dependency does play in. Adding 10s of megabytes and having to support any faults done by llvm is a huge undertaking.
That being said, I’ve been thinking that if we ever were to start working on adding more tiers to the JIT we should take a look at where LLVM is then and consider using it.
2 Likes
Thanks! The main reason I continue thinking about LLVM is that I’ve heard some “Erlang is not fast enough for our tasks” when I’ve been talking to people from near real-time domain, for instance, RTB in advertising. A motivation behind using Go instead of Erlang then is that Go is just more performant in terms of run-time speed.
When I asked about LLVM I had in mind also something more breaking than jit that uses LLVM, probably a case of AoT compilation with some kind of fat beam files with machine op code inside, wrapped into beam data structure. This is pure speculation of course, and I guess it’s quite far from the existing toolchain. The point is, however, that changes require budget and customers, but customers require a reason to build their use case with BEAM and feed the budget, and I guess it’s not very clear how to break this circle.