Yes, they are in a docker swarm in a cluster with three machines. In the image I got the atop from the docker and also from the host machine, with similar results in relation to beam.smp.
It is most likely a bug or a configuration error I made. I’m trying a few more tests, and any news I’ll post again.
I do not think so. The application does not make use of any advanced features. Just a job that runs every 5 minutes and does some checks. Then go back to waiting. The web area is accessed infrequently, just to access system settings.
Here’s next thing I would try myself. Run your app on the VM outside of docker. Might be a huge hassle, but it’s a process of elimination I might engage in if I was in your shoes.
If you don’t see the problem outside of docker, then we can say, this is related to docker and something needs to be tuned.
Interesting, it looks like you’re running the app via rebar itself vs making a release? That might explain a lot.
Everything indicates that the problem has been resolved. The ps aux shows some strange processes running, like two runs of ./srart.sh and one ./rebar.
I adjusted the swarm configuration to wait for a longer startup time start_period:60s. Also change the startup script to check if the application was already running.
CHECK_BEAM_IS_RUN=`ps aux | grep beam.smp | grep -v grep | awk '{print $2}'`
if [ ! -z "$CHECK_BEAM_IS_RUN" ]; then
echo "beam is running.."
exit 0
fi
Looks like the supervisor was trying to start the application more than once. Why it takes up the entire CPU I don’t understand, but after these changes the CPU consumption is less than 1% all the time.
Now ps aux show this:
To start the chicago boss in production is:
# Start Boss in production mode
echo "starting boss in production mode..."
START=$(./rebar boss c=start_cmd|grep -v "==>")
do_start "$START"
;;
As the terminal is released, I use the supervidord as a daemon.
I’m glad you resolved it!
Question, are you using rebar or rebar3 and just renamed it to rebar? rebar is legacy FWIW, you should be using rebar3. Likewise as I commented above, you should be using releases vs using rebar / rebar3 to start your app. As stated rebar is a development tool and running your app via rebar / rebar3 is going to be sub-optimal in many ways (performance being a big one!).
Another question, are you using supervisord to start the app or just docker swarm? The above reads like you’re using supervisord to start your app via rebar. FWIW, and especially when switching to releases, it’s usually best to have a service file for starting epmd separate from your app FWIW.
I have many other questions, I’m terribly interested in digging in further with you as to how you ended up where you’re at and how we the community can improve documentation and so forth to send new comers down a better path. I’ll pause there and wait for replies though 
Absolutely this! This is big.
It’s really rebar, not rebar3. By default the boss comes with rebar. I couldn’t run it with rebar3 after trying a bit.
The app run on docker swarm. Inside VM the app starts with ./start.sh script, which is called by supervidord. inside the script is executed the command ./rebar boss c=start_cmd|grep -v "==>"
Does it look confusing? it’s really confusing!
When I thought about developing a real project using erlang, I looked for web frameworks, and chicago boss seemed the most simple and complete, based on my knowledge. I focused more on the application and language than deploying techniques, so I used what the framework suggested. I’m still a little confused about the tools and settings needed to startup an application without the help of some framework. I know, I need to study and learn more.
Yeah, I just checked myself and it looks like the download and tutorial they provide comes with rebar and references it in their tutorial. This is a bit confusing to me as chicagoboss seems to be maintained, not sure why they’ve stuck with rebar vs rebar3.
I’ve never used chicagoboss, so I can not comment on it really. I’ve never used any erlang web frameworks in anger FWIW, only phoenix in Elixir, but if I had to pick one right now in Erlang I’d go with nova. The authors of Nova, such as @Taure, are active on here and in the community in general and I’m sure would be delighted to answer questions you may have.
Yeah, I mean, starting your app via rebar aside, I would not use supervisord to start things up inside a docker container, instead make your script the entry point for your container. I have other thoughts around docker usage, but that’s some what off topic.
I feel you! And IMHO, you shouldn’t have to think about some things, such as “What build / project tool should I use?”. I’ve never seen that scenario play out well (in other languages / communities). This is why I advocate rebar3 being part of erlang/otp (shipped with it). There should be good and clear getting started guide on erlang.org similar to what you will find on rebar3.org and in the adopting erlang book.
Not everything should be easy, but having a smooth and clear entry is a must IMO. Of course, that doesn’t solve chicagoboss using an outdated version of rebar and so forth, but do you think if some of this tooling and guides was part of Erlang/OTP and on erlang.org things would be more clear? Or do you think you would have ended up on chicagoboss github repo / website first regardless?
Other problem here is that the old version of rebar didn’t have support for relx (I believe) and required you to put together releases yourself via the tools included in Erlang/OTP. So if you stick with chicagoboss and such, you should read LYSE - Release is the word. .
Finally, yes you will have to study and learn, as is tradition, but getting started shouldn’t be so confusing or a mountain to climb.
I’d like to keep digging in with you and I’d implore you to keep starting threads and asking questions here to help you learn and so that we can learn how to improve the experience.
Hi,
Just to confirm. The application has normalized, and processing is pretty low now. Over the weekend I read adoptingerlang.org and am now reading the beam book. Thanks again for the help and reading tips!