We love erlang in situations, when you have to handle each socket for a very long time: hours, days, weeks. No other ecosystem can give that easiness of debugging leaking sockets, memory, etc.
Kubernetes, being a new operation system instead of linux or windows, brings new approach: no state, nothing persistent, everything temporary. Reconfigure by restart.
How can we deal with this in erlang? Seems that only way is to refuse from long living connections as much as possible. Move reconnect and retransmit logic to client side.
1 Like
At the moment, we do everything the k8s way, but we’ve been kicking around the idea of writing a BEAM-aware controller that would be used instead of the built-in deployment controller.
We haven’t got past the talking-about-it-over-beers stage, though. We also use a lot of Kafka, so our use case might not generalise very well.
Well, “talking-about-it-over-beers” is a very important stage, because it may be used to share experience. And just taking some beer is a good idea.
As far as I understand, the immutable nature of container doesn’t give a chance to mark it “upgraded” without full restart even if code migration was done properly. Correct?
But seems that you plan to postpone restart as far as possible with your own Operator?
1 Like
In a sense, yes. But the only thing that cares about whether a container was upgraded is the Pod object (and, by extension, the Deployment or StatefulSet that owns it).
So: if you make all Erlang pods use identical, maybe even empty, BEAM container images, then they don’t need to be upgraded.
At that point, you must use … something that needs inventing … to start the original applications and to track the state of the migrated applications within the BEAM.
Like I say: our use case is different from the general case. We don’t need to avoid restarts; we just need to manage them within a larger context. We might end up going in a different direction.
Remember that Kubernetes is just a bunch of shell scripts in a trench coat. Inventing your own resource types and controllers and operators isn’t that difficult.
1 Like
Yes, we write our own Operators, because it is required. I’m a bit afraid to fall into spaceship building heresy instead of practical things =)
There are some nuggets of useful info here (not sure how up-to-date it is though): Kubernetes | Adopting Erlang
1 Like