Hello – thanks in advanced for any advice here.
I’m interested in developing a chat application using vernemq as the message broker between clients.
I’m writing a plugin, which I want to handle auth as well as persisting the messages into a database for historical message retrieval.
I’ve been looking through the plugin docs as well as looking at other plugins, such as vmq_diversity and I have a few questions around how I’m supposed to make a resilient application using the plugin APIs. What I mean by resilient is that I need to ensure that my plugin hook eventually succeeds and that I’m able to control back-pressure and the responses to the clients during times of degraded health.
Let’s say I’m using postgres as my message store and also to store auth data.
My idea is to use the on_publish* callbacks to store messages. So I make an insert query and insert the message into postgres. This is all good in the happy path however the unhappy path is where my questions arise. It’s quite possible that at certain times the database may become overwhelmed and bottleneck verne. At this point I think I’d want to start sending PUBACKs with failure codes back to the clients. There’s currently no way that I can see to return errors from the on_publish* hooks. When an exception is raised, it appears the client is disconnected and the message is dropped (I think). I think dropping client connections because my database is swamped will just cause more turmoil on the backend. I’d rather just temporarily show the user the messages failed and keep the client connections alive.
I have the same question for the diversity plugin. If the database goes does will verne just shed all of the client connections until it’s back up (as soon as they try to publish and it starts throwing exceptions)?
Another solution I considered is to just subscribe to a wildcard for all the chat topics and use that to pipe the messages into a db. My concern there is that if the db gets backed up then those messages will also eventually be dropped and I’ll lose messages that way.