We've just released a new version of VerneMQ. This release contains a major refactoring of the queueing mechanism as well as several smaller improvements and bug fixes. Prior to this version the offline messages were stored in a global in-memory table backed by the LevelDB message store. A lack of flexibility regarding queue control was one of the consequences. There was no easy way to limit the maximum number of offline messages per client. This, and much more, is now possible. Unfortunately we break backward compatibility for the stored offline data. Therefore, you're required to delete the old message store folder /var/lib/vernemq/msgstore as well as the old metadata folder /var/lib/vernemq/meta. We apologise for this, but we guess it is the luxury for not being 1.0.0 yet ;)
We also updated the format of the exposed client metrics. Make sure to adjust your monitoring setup.
Prior to this version the offline messages were stored in a global Erlang ETS bag backed by the LevelDB message store. The ETS table served as an index to provide fast lookups for offline messages. Every client session was load-protected by an Erlang process acting as a queue. However, once the client disconnected the queue process had to terminate as well. With the new release, the queue process will stay alive as long as the session hasn't expired. This simplifies a couple of things: the routing mechanism doesn't have to distinguish between online and offline clients anymore, limits can be applied on a per client/queue basis, and we have gained more flexibility to deal with multiple sessions (a non-standard feature VerneMQ supports).
The current message store only relies on LevelDB and has no intermediate ETS tables for caching and indexing anymore. This improves overall memory overhead and scalability. Unfortunately this breaks backward compatibility and requires to delete the old message store folder.
The supervisor for the queue processes has much better performance for the setup and teardown of queue processes.
And as always keep the feedback coming, have fun!
The VerneMQ Mission Engineers