Feature: Counters/Atomics get/1 as list/tuple

Erlang/OTP Proposals Proposals: Ideas
1

I wanted to raise this potential feature and since it isn’t an “issue” I didn’t think OTP’s github was the right place to do it.

The need is for getting the value of every slot of a counter at once, as a tuple or list, so counters:get(Ref):

1> Ref = counters:new(3, []).
{atomics,#Ref<0.1669624794.3211395092.246705>}
2> counters:get(Ref).
{0, 0, 0}

I’m using counters to store the counts for an explicitly bound (so number of buckets remain the same) histogram and thus need to get each value when exporting the metric.

An issue is I think it requires iterating over each index in atomics so the get won’t be a snapshot of each counter at time T. Not sure if there is a way around this.

Hoping for feedback from the OTP team.

2 Likes
2

Non-yielding implementation is trivial. I was after this feature, but what stopped me, even when I have the counter list (tuple would be too large, we have thousands of counters exported), I still need to associate indices with counter names.

1 Like
3

Only if you run a single scheduler.
The implementation is totally lockless. Other scheduler threads may be updating atomics in the array while you are looping through reading them.
I think it’s possible, but not trivial, to implement a consistent snapshot by creating a new generation of the array for each snapshot.
The write_concurrency option may also complicate things further, as it then keeps one counter per scheduler and adds them together at read.

3 Likes
4

Suppose you do get a copy of all the counters in a group
“at the same time”. What can that be used for? Any/all
of the entries will soon be out of date. If the counts
are out of date anyway, why does it matter that they are
all “at the same time”.

So your question is about “CAN/HOW can I do this in Erlang”,
and my question is “WHY do you want to do this in any
programming language?” I’ve been puzzled by questions like
this for, oh, a couple of decades, when I noticed people
putting query operations in their concurrent interfaces
that were pretty much guaranteed to give out of date results.
I still have much to learn about concurrent programming, so
I look forward to being enlighted.

5

My case is a histogram where each counter in the counters ref is a bucket. So I’m exporting it at time T, any changes after T will be picked up by the next collection/export cycle.

6

I didn’t even think of a snapshot consistency. Pretty much the same reasoning @nzok provided: I didn’t need snapshot consistency for counters, because if they change so rapidly, it does not really matter if an extra counter bump is accounted for this or the previous minute.
I planned to use this for realtime event aggregation (counters - bumping, say, per request, error, or something else). Usually ets:update_counter is leveraged for that, but it’s way too slow, and requires striped ETS (or ETS per scheduler) approach for reasonable concurrency.

What I could not figure out is how to efficiently store (dynamically generate) counter names. So far the approach is to store the map of counter names → atomic array index in ETS table and periodically dump it to persistent_term (ecount implements exactly that).

1 Like
7

@sverker :

I think it’s possible, but not trivial, to implement a consistent snapshot by creating a new generation of the array for each snapshot.
The write_concurrency option may also complicate things further, as it then keeps one counter per scheduler and adds them together at read.

Something very similar to this functionality is already implemented in the VM (see erl_flxctr.h). In the implementation exposed by erl_flxctr.h, writes are as efficient as counters with the write_concurrency options, and reads get a consistent snapshots but with long latency. This functionality is currently only used for decentralized counters in ETS tables (used to respond to ets:info(T, size) and ets:info(T, memory) when decentralized counters are activated), but I think it would be easy to extend the counters API to expose this functionality as well. This blog post describes how this functionality works.

@nzok:

“WHY do you want to do this in any programming language?”

I agree that consistent snapshots are unnecessary for many applications, and it might be more desirable to get an answer fast than a “correct” answer slowly. However, there are some situations where it is useful to get a snapshot of an array of counters. For example, the ETS counters described in the article I refer to above uses an array of counters under the hood, and without “snapshots” users could get strange results (e.g., a negative items count), which would not be very user-friendly.

2 Likes
8

This does not sound like absolute consistency would be crucial. Especially if your buckets contain sums of lots of small increments.
If bucket[2] is updated before bucket[9], is it really that important to read the updated bucket[2] if you read the updated bucket[9] when collecting the histogram?
You could of course get really unlucky with the OS deciding to yield your histogram reading thread in the middle of the loop. Then you might get some milliseconds worth of inconsistency.

1 Like
9

Yea, I don’t think it is absolutely necessary. It is just technically not guaranteed to be a consistent snapshot of the histogram at time T.

1 Like
10

Much depends on what is being counted and how the events are
reported. If the report of an occurrence of event 1 arrives
at the counter through route 1, and the report of an occurrence
of event 2 arrives at the counter through route 2, and route 1
and route 2 have different and possibly varying delays, what
does it even mean to have a consistent snapshot of the counts?
“Route” here includes whatever happens inside an Erlang node,
including process scheduling. I remember people working on the
Square Kilometre Array project complaining bitterly about jitter
in the Linux kernel; that was a situation where “timelike slice”
had a real physical meaning that was important to preserve.
And that’s why they were building very very specialised hardware
to preserve it.