Keep both “latest” (usually called “active”) and “old” in the same table, wrapping the original key in a tuple of {TabName, Key}. It’s also often extended to the concept of the “striped table”.
Store active table name in the persistent_term, so fetching data from the cache looks like ets:lookup(persistent_term:get(active_table_name), Key). When you’re about to swap the tables, just call persistent_term:put(active_table_name, NextTableName).
Of course both approaches are easy to generalise.
Unfortunately there is no atomic swap operation swapping two names ETS tables. It might be worth adding it, it appears to be a common ask for OTP.
You must already have a process that owns the tables and handles the turnover, right? So what I would do here is try to query Tab1. If that fails because Tab1 is currently missing, meaning turnover is happening just at that moment, send a message to and wait for a reply from the owning process. The owning process should (receive and) reply to such messages only after it has finished the turnover, meaning Tab1 will be there again, and the querying process can try accessing Tab1 again.
Depending on how the cache is used (query frequency, turnover frequency, …), this waiting may actually defeat the entire purpose of having a cache in the first place, so it could also be possible to just proceed as if the queried value was not present in the cache at all on failure.
How to make it look atomic, and generalised to more than one table? Keep an atomic in a persistent term that points to the current table to start from. Then querying processes will read that atomic and cycle through the ets tables starting at the index the atomic points to. A background process periodically increments the atomic (in a rotatory way), and cleans up whatever was there in the table that is now considered last after rotation.
)