What is the better practice when designing MQTT topics for thousands of devices of multiple customers?

I am developing a MQTT service and I am confused when I design the topics. I think there are two ways to do this:

  1. Topic data/customer-a
    Payload {"sensor-id":"23aafff2d23659cc97c557909de12f16","type":"demo","value":"hello world"}
  2. Topic data/customer-a/23aafff2d23659cc97c557909de12f16
    Payload {"type":"demo","value":"hello world"}

I wonder which is better. I think the first one may result in complexity when I need to subscribe message from a single sensor. Meanwhile, the topic data/customer-a is always under heavy load because of thousands of messages per second. And the second one has drawbacks too, obviously it makes the mqtt broker maintain a huge topic tree, I don’t know whether it would cause significant performance degradation?

Which is better for thousands of devices of multiple customers?

1 Like

I would design it like /devices/{UUID}/data and check the scope/access to the topic on authentication/authorization hooks without exposing the customer names or ids in the topics. Unless you have a use case to use a wildcard to subscribe to all the devices of a customer at once.

2 Likes

We do it in a similar way in zotonic. Name from less to more specific. E.g. model/rsc/event/{ ResourceId }/{ insert|update|delete} for inserts, updates and deletion of resources. Similar for edges between resources. model/edge/event/{ SubjectId }/o/{ PredicateName }.

And about the load, this entirely depends on the implementation of the router of course. We use an ets based trie data structure to map message topics to subscribers. So it doesn’t really make a higher level names more loaded or something, it is just an handy way of routing the messages.

3 Likes

thanks a lot for your advice and respect for your work. But I am still confused whether too many topics would impact on vernemq service performance? chould you please give me some guidance?

1 Like

This probably depends more on the number of subscribers you have than the number of topics you use in your messages.

4 Likes

VerneMQ uses a trie. I have personally noticed more performance degradation from having many subscribers than for having many topics. But every use case is different, so load testing with your specific topic structure, number of devices and specific message sizes is what will give you a good idea of the performance. No load test is good for every single case, Load testing VerneMQ, the basics might be of help to you.

3 Likes

@codeadict @mmzeeman yes, you are right, the number of subscribers is the real key point. Thank you all guys for giving professional advice and contribution to the open source community!

3 Likes