Ets memory occupy - the erL shell query occupies more than 2M memory, but the observer query occupies more than 18M memory

Questions / Help
1

The following operation creates an ETS table and inserts 300,000 items of data.

Blockquote
T = ets:new(tttt, [named_table]).
_ = [ets:insert(T, {O, O}) || O ← lists:seq(1, 300000)].
ets:info(T).
[{id,#Ref<0.2452677512.3532259330.103184>},
{decentralized_counters,false},
{read_concurrency,false},
{write_concurrency,false},
{compressed,false},
{memory,2401625},
{owner,<0.87.0>},
{heir,none},
{name,tttt},
{size,300000},
{node,‘aa@192.168.2.43’},
{named_table,true},
{type,set},
{keypos,1},
{protection,protected}]

Then the erL shell query occupies more than 2M memory, but the observer query occupies more than 18M memory. From the perspective of the operating system, it also occupies more than 18M memory

Feels like a lot of wasted space

2 Likes
2

I think the 18M of memory you are seeing are from the list comprehension and are held by _.

In the erlang terminal, after using _, you can still refer to the value by the line id (2> in the line below):

2> _ = [ets:insert(T, {O, O}) || O <- lists:seq(1, 300000)].  
3> v2().
[true,true,true,true,true,true,true,true,true,true,true,
 true,true,true,true,true,true,true,true,true,true,true,true,
 true,true,true,true,true,true|...]

Can you check if you see the same results with:

lists:foreach(fun(O) -> ets:insert(T, {O, O}) end, lists:seq(1, 300000)).

This line of code doesn’t hold the return value of ets:insert in a variable so it won’t appear in observer.

2 Likes
3

There is also the option f() in the shell that will delete all the bind variables Erlang -- shell

2 Likes
4

1
11279×617 36.9 KB

  • The result is the same, I guess ETS has a mechanism for pre-allocating memory
1 Like
5

Sorry, I didn’t realise before. The memory in ets:info(Tab) is in words and not in bytes.

From the memory page:

…A word is therefore 4 bytes or 8 bytes, respectively. The value for a running system can be determined by calling erlang:system_info(wordsize).

Therefore, assuming your implementation uses a word size is 8 bytes the total number of bytes is 2401625 * 8 = 19213 kB.

It doesn’t totally match the observer value but it is not 2MB as you thought in the first comment. It would be interesting to know why it shows a different value.

2 Likes
6

oh, it is the words， That’s right. 2401629 * 8 / 1024 = 18762.7265625 KB

thanks you all

2 Likes
7

Then observer should show MiB instead of MB :smile:

2 Likes
8

:grinning: :grinning: :grinning:…

1 Like