Hello,
This is more sharing and also asking opinions as I don’t have the full context. So, it’s better to ask people 
In Erlang/OTP source code there is a hidden build defined called SMALL_MEMORY. It is marked
as undocumented and unsupported SMALL_MEMORY feature. Reading the code it could be interpreted as “pre-allocated memory is used, more aggressive gc, and smaller time tables”.
I tried to check erlang source code in order to get an idea (mainly related to erlang memory allocators) how much SMALL_MEMORY does. Here is some results/interpretation:
Looking into the code and after trying to mimic SMALL_MEMORY configuration using erl parameters I ended up with the following:
# From here and below it tries to mimic small_memory (-DSMALL_MEMORY) behavior
EXPERIENCE+=" +Muycs 128 +Mummc 1024"
# temp T
EXPERIENCE+=" +MTrmbcmt 100 +MTrsbcmt 80 +MTrsbcst 90 +MTsbct 64 +MTmmbcs 32 +MTlmbcs 1024 +MTsmbcs 128 +MTramv true -MTt true"
# literal I
EXPERIENCE+=" +MIrmbcmt 0 +MIrsbcmt 0 +MIrsbcst 0 +MImmbcs 256 +MIlmbcs 1024 +MIsmbcs 128 +MIramv true -MIt true"
# ll L
EXPERIENCE+=" +MLrmbcmt 0 +MLrsbcmt 0 +MLrsbcst 0 +MLmmbcs 1024 +MLlmbcs 1024 +MLsmbcs 128 +MLramv true -MLt true"
# ehep H
EXPERIENCE+=" +MHrmbcmt 50 +MHrsbcmt 80 +MHrsbcst 50 +MHsbct 64 +MHmmbcs 256 +MHlmbcs 1024 +MHsmbcs 128 +MHramv true -MHt true"
# std D
EXPERIENCE+=" +MDrmbcmt 50 +MDrsbcmt 80 +MDrsbcst 20 +MDsbct 64 +MDmmbcs 32 +MDlmbcs 1024 +MDsmbcs 128 +MDramv true -MDt true"
# sl S
EXPERIENCE+=" +MSrmbcmt 50 +MSrsbcmt 80 +MSrsbcst 80 +MSsbct 64 +MSmmbcs 32 +MSlmbcs 1024 +MSsmbcs 128 +MSramv true -MSt true"
# binary B
EXPERIENCE+=" +MBrmbcmt 50 +MBrsbcmt 80 +MBrsbcst 20 +MBsbct 64 +MBmmbcs 32 +MBlmbcs 1024 +MBsmbcs 128 +MBramv true -MBt true"
# ets E
EXPERIENCE+=" +MErmbcmt 50 +MErsbcmt 80 +MErsbcst 20 +MEsbct 64 +MEmmbcs 32 +MElmbcs 1024 +MEsmbcs 128 +MEramv true -MEt true"
# driver R
EXPERIENCE+=" +MRrmbcmt 50 +MRrsbcmt 80 +MRrsbcst 20 +MRsbct 64 +MRmmbcs 32 +MRlmbcs 1024 +MRsmbcs 128 +MRramv true -MRt true"
# fix F
EXPERIENCE+=" +MFrmbcmt 50 +MFrsbcmt 80 +MFrsbcst 20 +MFsbct 64 +MFmmbcs 128 +MFlmbcs 1024 +MFsmbcs 128 +MFramv true -MFt true"
But, later I realized that the above parameters does not make sense since for SMALL_MEMORY as there is a ifdef disabling allocators:
#if defined(SMALL_MEMORY) || defined(VALGRIND) || defined(ADDRESS_SANITIZER)
#define AU_ALLOC_DEFAULT_ENABLE(X) 0
#else
#define AU_ALLOC_DEFAULT_ENABLE(X) (X)
#endif
That means when SMALL_MEMORY is active (for erl_alloc perspective), it’s like there are no allocator, thus the system behaves
like small memory when checking OS RSS for instance. Moreover, SMALL_MEMORY does not give much when talking about memory fragmentation and performance when allocating memory. Because calls to malloc will be increase when SMALL_MEMORY is enabled.
In a system without SMALL_MEMORY enabled, I think, using the option +Mea min could give the same results (related to memory allocators).
Continuing tracking down the SMALL_MEMORY define usage and besides erlang allocator, there are more places where SMALL_MEMORY is used:
erts/emulator/beam/erl_hl_timer.c:#ifdef SMALL_MEMORY
erts/emulator/beam/erl_process.h:#if defined(SMALL_MEMORY)
erts/emulator/beam/erl_time.h:# ifdef SMALL_MEMORY
- erts/emulator/beam/erl_process.h, I could not get where the defined MBUF_GC_FACTOR is used. So, I believe it’s not used at all. This can clearly be removed
- erts/emulator/beam/erl_time.h and and erts/emulator/beam/erl_hl_timer.c time size and time wheel related.
I could not estimate how much memory a system saves when SMALL_MEMORY is active for time table related. So, I believe it saves some KB.
Looking into git logs I could not identify why SMALL_MEMORY has been implemented, which issues it was trying to solve and mainly if that flag still makes sense and worth keeping it.
In fact, when checking old erlang releases like 17 or even from GitHub - mfoemmel/erlang-otp: All of the public Erlang/OTP source releases (since R6B-0 in 1999) in convenient git form R13B02 it was possible to understand that in the past SMALL_MEMORY had changed a few more places. But when looking into erlang/otp master branch, these changes are not true anymore.
And you, have you ever knew about SMALL_MEMORY ? Have you used it before ?
I know it is undocumented and unsupported. And maybe it should be removed anyway ? As one could get the very same effects using only standard erl parameters.
Thanks
