Arguments for Transitioning to Erlang

At work, we have a project with a monolithic architecture written in C++. It has become unwieldy. We have decided to refactor the project into something more modular, maintainable, scalable, fault tolerant, and distributable.

Because Erlang was designed to solve these types of problems, I want us to incrementally rewrite the program in Erlang. However, I’m having a hard time convincing the rest of the team and my manager. Their biggest concerns revolve around training our team in a new language, being able to hire Erlang developers, and the lack of “popularity” of the language.

How have you addressed these concerns? Are there examples of companies that have made the transition that I can use to support the migration?

5 Likes

I believe @rvirding has trodden this path before, albeit quite a long time ago!

5 Likes

Joseph,

I run an Erlang shop and what I have done is to turn the problem around. I hire new grads with little experience and train them. They’re cheaper to hire, I’m not paying for the experience I don’t want (e.g. Java), and getting fresh, uncorrupted, minds ready to learn the Erlang philosophy as primary. Admittedly I do this in a developing nation which makes it more affordable but I think the basic premise applies everywhere.

9 Likes

As @phild said, if they were referring to the axe-n project collapse at Ericsson and how Erlang came to the rescue, @rvirding would indeed know a good bit. Mike Williams covers at least a bit of that here in this talk :

The True story about why we invented Erlang

And here’s a talk (albeit old) by Ken Wilson on moving a team from C++ to Erlang :

Migrating a C++ team to using Erlang to deliver a real-time bidding ad system: Ken Wilson

As far as Erlang being a non-mainstream language and the myth of not being able to find developers, well, I think I said it, it’s a myth :slight_smile: As @vances said, hire folks who have fresh minds, but also a good developer can pick up anything, and that’s what you really want to hire in the end IMO.

I’m sure there’s many more cases out there and I’ll dig to see if I can find more myself, and specifically around migrating C++ teams to Erlang.

I can tell you one thing I did at a company (this was not a C++ shop) to convince them to get on the BEAM was showing them the properties that you alluded to, but showing them how poorly their existing system utilized their 64 core server(s) vs a prototype I wrote which beautifully utilized all cores, that pretty much did it. That and sharing actual cases with them (e.g., WhatsApp), particularly around ROI. In addition though, it was also about the language itself, Mike Williams touches on this a bit in his talk as well :slight_smile:

8 Likes

Some great posts already :023: I was going to say something along the same lines as Bryan - what kind of app/project is it? If you can let us know maybe we can mention some companies in the space already using Erlang (or another BEAM language) :003:

3 Likes

Yeah, thanks for asking that question. Context is king!

3 Likes

Thank you for your responses, they have been helpful. I especially appreciated the talks you posted @starbelly. For context, this application is an intrusion detection system (IDS).

3 Likes

That’s a niche space! I’m willing to bet some folks closer to this space might be able to shed some more light on this area.

I’m not sure it helps, but I had to build something similar to such a thing recently (but not for IDS purposes), but involved capturing packets off a NIC and reassembling the stream, etc. I can’t even imagine doing this in another language! The SLOC compared to what I saw written in other languages (including C++) was nothing in comparison and all quite readable IMO.

Doing this in Erlang (in my case it was actually Elixir, but it’s all erlang!) was a breeze. If it helps check out pkt and epcap

I think we’ll be able to do a lot of that via OTP itself (specifically the socket api) soon FWIW, thus perhaps in the not so distant future you wouldn’t even need external libs.

Edit:

Interesting paper Embarking on the road of Intrusion Detection, with Erlang

Edit:

I put ecpap in the above, I actually used ewpcap.

5 Likes

Nice find Bryan!!!

Great summary too:

In this paper we have shown that the Erlang programming language can be used for developing IDS that exhibit robustness and scalability. Erlang’s features such as hot code loading allow updating the IDS without affecting the current operation of the system. The supervision tree allows failed components to be restarted. Concurrent distributed processing is another feature useful when using AIS as a basis for IDS. While the processing power of an Erlang node is smaller than that of C based firewall, the Erlang system takes better advantage of multi-core architectures and provides easier application development due to its functional nature.

I also found the below from this talk about BUILDING A HOME SECURITY SYSTEM WITH ELIXIR AND NERVES by Arto Bendiken - YouTube :smiley:

4 Likes

Very cool :sunglasses:

2 Likes

Hi Joseph. I’ve been countering these arguments for 30 years. Start the other way around. How many people evaluate and choose the most appropriate technology? It sounds like one of the Erlang languages does indeed suit your needs. As Vances says, it’s not that hard to find Erlang practitioners. Anyone who has studied or used a functional language can pick it up. There are many good arguments to enable the transition. One is that Erlang was designed to interface easily to other languages, so you don’t have to re-write everything at once. WIth a highly cohesive monolith, you can divide it with force, using RabbitMQ to separate functions, then re-write the parts separately.

4 Likes

Some anecdata regarding training. We had a situation a few years ago where several of our major components were written (by me) in Erlang, but the larger team didn’t know it. We had some training money available, and had Erlang/OTP training suitcased in. We had about a dozen good developers, all experienced in Java and/or Python or other languages. Nobody knew Erlang but me.

We started Monday morning setting up a lab and introductions and all of that. By Wednesday noon we had covered the whole language and started into OTP. Friday afternoon the group built a working, though rudimentary, distributed chat system. Got it up and running and still had time for everyone to catch their flights home. It was pretty impressive.

6 Likes

That’s awesome Garry - did you do the training yourself? If so might be worth making it into a small course or something similar :003:

1 Like

No. We used Erlang Solutions to do the training. They did a great job.

4 Likes

Thank you to everyone, your feedback has been very helpful. I’ve spent the last week opening the team up to using Erlang. They’re not 100% on board, but opened up to moving down that road. I’m now working on convincing the program manager and stake holders.

8 Likes