In this first blog post on MQTT 5 we'll take a look at the features of the new protocol version and consider the costs of moving to it.
The MQTT version 5 was released about 6 months ago and a lot of people have been blogging and tweeting and talking about the merits of this protocol update. And there are tons of improvements! Here's a list of some of them:
Metadata via user properties: the ability to add extra information which can be inspected without having to parse and inspect the payload.
Provisioning features where brokers can tell clients to (re)connect to other broker instances.
Negative acknowledgements which makes it a lot easier for a client to understand what's happening.
Shared subscriptions to easily handle massive fan-in scenarios.
Topic aliases which saves band-with when publishing messages.
Use Message expiration to prevent clients from receiving stale/irrelevant information.
Differentiate received messages for which the subscriptions overlap using subscription identifiers.
Fine grained retained messages handling.
Request/Response interactions.
Enhanced authentication mechanism.
All these features are sorely needed. Some more than others. Negative acknowledgements allow for much easier debugging cycles when trying to understand why some client can't connect, publish or subscribe to a specific topic by telling the client what actually happened. Shared subscriptions are needed to overcome the limitations of a single subscriber receiving more messages than it can handle. Topic aliases are great for those clients where the up-link is expensive and saving as many bits as possible is paramount.
If any of these features are important enough to warrant switching to MQTT 5 of course depends on the use case. In general, though, one can consider if the new features makes things more complicated for the application developer or if they make the protocol itself less lightweight.
The protocol is very much still the same. The only new packet is the AUTH packet which can be used to support SASL type authentication schemes. The rest flow as usual and the only case where more packets are flowing compared to MQTT 3.1.1 is to support the new features such as negative acknowledgements where earlier the client would just be disconnected.
The biggest change on the protocol level is the addition of properties which (for most) packets are optional. Properties are a list of key-value pairs which is used to support many of the new features introduced in the protocol. For instance topic aliases, message expiration intervals, enhanced authentication methods and data, and metadata (user properties) are all implemented as specific properties on the various packets.
On the wire this means that packets can contain extra optional data that only has to be parsed if it is there. The additional extra computational overhead is therefore to check if optional data is present or not, and if so, to parse it.
The extra work the parser has to do is extremely little and would not be noticeable on small scale MQTT deployments and even on large scale deployments it would not be expected to have any significant impact.
If the new features are used and for instance all publish messages carry a number of properties that needs to be parsed and handled this will be more costly. But the decision if using the feature and tolerating the associated computational cost is luckily up to the application designer!
So MQTT 5 basically provides a lot of new features while making sure the computation and band-with cost is effectively zero if they aren't used!
From the perspective of the application developer using or designing MQTT 5 applications it is pretty much business as usual. Except, of course, that using some of the new features make some applications simpler as a lot of the logic found in applications is now handled by the brokers and client libraries.
One example is the shared subscriptions feature which makes it possible for subscribers to a specific subscription to share the subscriptions between them. The goal here is of course to handle massive fan-ins which would a single client would otherwise not be able to handle. In MQTT 3.1.1 this would be hard to implement without some kind of name-spacing or sharding to leak into the topic structure which can make it less intuitive and harder to reason about.
Another example is the Request/Response pattern. Even though it is in a sense a slightly unusual pattern on top of a Pub/Sub paradigm, sometimes it is necessary for some use cases and each time it is needed it is usually implemented in a slightly different way. MQTT 5 therefore defines a standard way how to do Request/Response interactions using properties.
And this, I hope the authors of the MQTT 5 spec agree, is one of the major points of the new protocol version: to use the lessons learned how MQTT is used and support the most important and common patterns directly to make hard things simpler!
Of course, it's not all peaches and plums. Compliant client libraries and brokers have to implement all these new features in all their glory and complexity. Fortunately most people don't have to worry about this as a bunch of great MQTT 5 compliant client libraries are already in development! For instance the Eclipse Paho C, Go, and Java clients are well under way and the highly popular Mosquitto broker has also announced MQTT 5 support by August this year.
Of course VerneMQ is no exception! We've been working hard on implementing the full MQTT 5 spec. A developer preview of VerneMQ with MQTT 5 is currently available in the mqtt5-preview branch but we're working on moving the functionality into the master branch shortly and then it will be generally available in the nightly builds and coming VerneMQ releases.
So the answer to the title question is: yes, we absolutely do believe it is worth the effort to migrate to MQTT 5. The costs are negligible and the benefits are huge!
Of course adding MQTT 5 support to VerneMQ doesn't mean we're taking focus away or lessening our commitment to MQTT 3.1.1 support in VerneMQ and, in fact, the two MQTT protocol versions will work seamlessly together. Furthermore we've been working on making it possible to re-use MQTT 3.1.1 plugins in an MQTT 5 context, to make the transition easier. But that's a topic for another blog post.