Server for writing and API for concurrent access to ETS tables: architecture (and testing) question

Questions / Help
,
1

Hello!

I have been learning Erlang, and decided to build a toy interpreter as an exercise.
Then I got confused when trying to implement a symbol table… See what I’ve done so far:

  • The symbol table is implementer as an ETS table.
  • The interpreter should work with several processes reading and writing the table, so there is one writer (gen_server behavior) which handles all writes and an API for concurrent processes to read and write to the table (a simple module). This API will call the server for writes, and directly read from the table (if N processes try to write, they’ll naturally be serialized because they all call the same server; if N processes try to read, they will use the API simultaneously).

So the problem is… There are two modules (the writer and the symbol table API), and when I started writing the tests, I got stuck: I can;t test the write server alone, so there are these options:

  1. Write tests for the write server and check the effect of each message by directly reading the ETS table (but this makes the test implementation dependent): test things like gen_server:call(symbol_writer,{sym_intern, "symbol_name"}), and then do ets:lookup directly to check the table

  2. Write tests only for the API, and the server will be indirectly tested (but sounds like a bad idea for testing): test SymbolId = symbol_table:intern("symbol_name") and then check with ets:lookup(symbols, "symbol_name").

  3. Write tests for the server, and use the API to check the effect of the writing messages on the table (but testing the server would depend on an external module – the general symbol table API). gen_server:call(symbol_writer,{sym_inern, "symbol_name"}), and then check with ets:lookup(symbols, "symbol_name").

Is the architectural approach OK (having the server do only the writing, to allow for concurrent reading, and do all the interaction through an API)?

If the architecture is OK, how would be the best approach to testing?

Thanks a lot!

2

My suggestion is

  1. Write black box tests using only the API.
  2. Question if you really need that server process to serialize writes.
  3. If answer is “no”, remove the server and rerun your tests unchanged.
3

I suppose it’s necessary to serialize access, since it’s a tiny Lisp interpreter, and symbols should be unique. Creating a symbol with a name should not be possible for two processes (and the creted symbol should have a single ID). As I understand, ETS tables do not support that (right?)

4

If the symbols are the table keys, then you can use ets:insert_new that will fail if the key/symbol already exists in the table.

5

Ah, that will work, thank you!

If I ever need two keys per symbol (both name and a unique number) then this won’t work anymore, right? (Having previous experience with databases, my first idea would be to create two tables, one with the name as key and another to simulate a SQL index – mapping numbers to names).

6

Well, ETS have no general support for transactions. But there are some limited guarantees of atomicity and isolation.

  • All read and write operations to a single tuple.
  • insert and insert_new with a list of tuples. They will either insert all tuples or none.
  • delete_all_objects
  • select_replace can do a read-modify-writeback operation that is atomic for each tuple.

To do more advanced “transactions” a write server can be a solution. But for your specific example you could put both symbol names and numbers in the same table and do a single atomic insert_new operation:

ets:insert_new(T, [{Symbol, Number, Data}, {Number, Symbol}]).

Assuming that symbols and numbers can never clash due to different types.