Saving band-with with topic aliases

One of the new features introduced with MQTT 5.0 is the concept of topic aliases. When publishing MQTT messages the topic is, by necessity, always included so the broker can route the message appropriately. For small message payloads the publish topic can easily make out the biggest part of the complete packet, and naturally in settings where bytes are expensive this seems wasteful. The idea of topic aliases is simple: use integer identifiers which can be used in place of the longer topics.

MQTT is often seen as the IoT protocol because it is lightweight and has a small footprint both in terms of band-with and processing resources. And exactly for this reason it is often used as the communication protocol for small embedded devices with very limited resources. The topic aliases feature is perfect for these uses cases and makes it possible to save band-with as well as memory, which can be expensive resources.

In this post we'll show an example of how to use topic aliases when publishing messages to VerneMQ, though, it should be noted, VerneMQ also supports broker to client topic aliases. We'll be using VerneMQ 1.9.2 as well as the mosquitto_pub and mosquitto_sub command line tools version 1.6.7.

Assume we have a client with the client-id 0123456789ABCDEF and it needs to publish a QoS 0 status message to the topic status/0123456789ABCDEF (23 bytes) with a payload of {"status": "ok"} (16 bytes).

To configure VerneMQ for topic aliases we'll add the following lines to the end of the vanilla vernemq.conf file:

allow_anonymous=on
listener.tcp.allowed_protocol_versions = 5
topic_alias_max_client = 10

The first line will allow clients to connect without having to do any kind of authentication (only recommended for testing!). By default VerneMQ comes with the vmq_acl plugin enabled, which will by default allow any client to subscribe and publish to any topic. The second line configures the default MQTT listener running on port 1883 to accept only MQTT 5 connections and finally the third line configures that the client is allowed to use topic aliases 1 through 10 - if set to 0 clients are not allowed to use topic aliases.

The way topic aliases work is that when a message is first published, then the client will alongside the topic send a topic_alias which is then used when the client wants to publish to the topic later on.

Using mosquitto_pub to publish two messages then looks like this:

$ mosquitto_pub -i 0123456789ABCDEF -t status/0123456789ABCDEF -V 5 -D publish topic-alias 1 -l -d
Client 0123456789ABCDEF sending CONNECT
Client 0123456789ABCDEF received CONNACK (0)
{"status": "ok"}
Client 0123456789ABCDEF sending PUBLISH (d0, q0, r0, m1, 'status/0123456789ABCDEF', ... (16 bytes))
{"status": "ok"}
Client 0123456789ABCDEF sending PUBLISH (d0, q0, r0, m2, '(null)', ... (16 bytes))

In the above we see the second publish has '(null)' for the topic as expected.

Using the vmq-admin trace tool to trace the publish client the above exchange looks as follows from the broker perspective (edited for brevity):

$ vmq-admin trace client client-id=0123456789ABCDEF
New session with PID <9423.713.0> found for client "0123456789ABCDEF"
MQTT RECV: CID: "0123456789ABCDEF" CONNECT(c: 0123456789ABCDEF, v: 5, u: undefined, p: undefined, cs: true, ka: 60)
    with properties: #{p_receive_max => 20}
MQTT SEND: CID: "0123456789ABCDEF" CONNACK(sp: 0, rc: success(0))
    with properties: #{p_topic_alias_max => 10}
MQTT RECV: CID: "0123456789ABCDEF" PUBLISH(d0, q0, r0, m0, "status/0123456789ABCDEF") with payload:
    {"status": "ok"}
    with properties: #{p_topic_alias => 1}
Calling auth_on_publish_m5(undefined,{[],<<"0123456789ABCDEF">>},0,status/0123456789ABCDEF,false) with payload:
    {"status": "ok"}
Hook returned "ok"
MQTT RECV: CID: "0123456789ABCDEF" PUBLISH(d0, q0, r0, m0, "") with payload:
    {"status": "ok"}
    with properties: #{p_topic_alias => 1}
Calling auth_on_publish_m5(undefined,{[],<<"0123456789ABCDEF">>},0,status/0123456789ABCDEF,false) with payload:
    {"status": "ok"}
Hook returned "ok"

Here we see the same, the first publish includes the complete topic and the topic alias and in the second publish the topic alias is included and the topic is the empty string.

So with this example, how many bytes were saved with the 16 byte payload {"status": "ok"}? The size of the complete TCP payload for the publish message without topic aliases is 44 bytes. Using topic aliasing the TCP payload size of the first publish is 47 bytes as that includes the topic alias and the topic and for the second publish it is only 22 bytes. So when topic aliasing kicks in we save 50% of the bytes and the overhead per payload goes from (44-16)/44*100=64% to (22-16)/22*100=27%.

No topic aliasing With topic aliasing

Conclusion

Using topic aliases makes it possible to save significant band-with. In the concrete example we saw that the payload to metadata ratio went from 64% to 27% and the absolute number of bytes went from 44 to 22 when topic aliasing was being used. The first packet used to set up the topic alias was 47 bytes, so in this case topic aliasing already pays for itself if more than two messages are published to the same topic as 44 + 44 = 88 is larger than 47 + 22 = 69.

MQTT 5 performance