On some virtualized machines (possibly a bit exotic), some Erlang programs were aborting due to the following problem: “Monotonic time stepped backwards!”.
Adding the “+C no_time_warp” flag apparently solved the problem for these programs, yet their compilation was also affected by the very same problem (erlc then aborting).
In addition to erl, is there a way of specifying the aforementioned option to erlc? (just specifying that option as it is results in “bad term: C” - as it could be anticipated by reading erlc’s documentation)
After reading Erlang -- Time and Time Correction in Erlang, I thought that the “+C no_time_warp” option could ensure (possibly at the risk of freezing a bit) that the Erlang VM time remained silently monotonic, despite the OS one going possibly backwards at times; nevertheless (and after having double-checked) a command-line like (with Erlang/OTP 26/erts-14.1):
Monotonic time stepped backwards!
Previous time: 1806968038
Current time: 1806967880
Aborted
(I could check that erlang:system_info(time_warp_mode) returned ‘no_time_warp’ indeed)
I was wondering whether it was among the expected behaviours and if there was a way of avoiding this problem as a whole (even if necessary by sacrificing the performances - for us it is just for the sake of a demo; we just want it to continue instead of crashing).
The runtime system aborts with Monotonic time stepped backwards! if it detects that the operating system’s monotonic time stepped backwards which should never happen. If it does, the clock is broken and cannot be used for time correction. If you disable time correction (+c false), it wont use OS monotonic time and should not abort this way. You can read more about how this effects time in the runtime system here.
Thanks Bryan and Rickard; I think I figured out how this could happen: our user node spawns and connects to a (here: single, local) computing node, and I guess the monotonic problem was finally happening only in the latter one, to which the non-timewarping option had not been propagated; the time error must have been forwarded to the user node (I imagine thanks to some group_leader related mechanism), which was reporting it. At least that is what I imagine is happening.
That said, the reason I asked what OTP version you were using is that applying no_time_warp might be something of a noop if you’re running Erlang/OTP < 26 then no_time_warp is the default anyway and you are probably just lucking out. If you really want to avoid a crash in a demo then you ought to follow Rickards advice and disable time correction all together, until such time you can figure out why os monotonic on your VM is jumping backwards, which as Rickard stated should never happen.
Hi Bryan,
Actually I had specified (through my secondary, work-related, forum account) that it was Erlang/OTP 26/erts-14.1 (that is 26.1; I prefer sticking to the latest stable version).
For the sake of this demo, our application is running on a Debian 12 (Bookworm) virtualised (thanks to VirtualBox I imagine) on top of a Windows host (of course we would have preferred a more suitable setting), and apparently this combination may result in non-monotonic time.
I anticipate that a no_time_warp option - this time applied to all nodes involved - will suffice; otherwise, as Rickard mentioned, disabling time correction (+c false) will still be an option.
Time will tell! (pun intended)
Hi Alexander,
Thanks for your message. Actually it is just a one-off demonstration (for which performances do not matter that much), so I guess I will not investigate further, now that time-warping settings are propagating automatically from the user node to the spawned ones (to be tested soon in real conditions).
The application is intensely concurrent but anyway meant to run on a proper platform (not virtualized - especially not on top of a Windows OS!).
Best regards,
Olivier.