Documentation for eMQTT5 client

Introduction

eMQTT5 is a MQTT client for protocol version 5.0 written in C++.

It's main purpose is to provide a lightweight client both in term of binary size, memory footprint and code size that's implementing as much as possible of the standard.

It is cross platform, and expects an hardware abstraction layer (HAL) to be written for network communication. Two examples of HAL implementation are provided in the code base, one using BSD's socket API (with mbed_tls library for TLS encryption) and the other using a high performance code (but less lightweight) derived from ClassPath.

Concepts

MQTT is roughly a messaging system where packets are formated following a specific serialization protocol. You can not send anything via MQTT unless you apply to MQTT's specific serialization rules. The protocol defines 16 packet types and the sequencing of their generation in a communication.

This library does the low level work of generating the packet as expected by the standard, and respecting, as much as possible, the expected sequence for each packet.

It abstracts the complexity of the protocol (like the initial connection with authentication, the Quality of Service management while publishing and receiving packets, the payload serialization and many more).

From your application point of view, you'll typically have to:

  1. Connect to the MQTT broker (opt. authenticate)
  2. Subscribe to some topics
  3. Publish some messages
  4. Run the event loop for as long as you want to receive packet on the subscribed topics
  5. Disconnect from the server

No memory copy

A great care has been spent on avoiding copying data around. This means that the RAM usage will be minimal since your data will be used as-is, and you'll be able to read the received packets directly. The library is using the concept of View (like in C++ string_view) over your data where it's possible to do so.

Using Properties with the client

Since this library is oriented for embedded usage, a great care was taken for avoiding heap usage and minimizing code size.

Packet in destination of the broker

In order to acheive theses goals, the Properties class is a chained list where each node stores a flag telling if it was allocated on the stack (default) or on the heap.

When the chained list is destructed, each node will either suicide(delete itself if heap allocated) or just chain the destruction request to the next node.

Parsing the chained list is done recursively, but this shouldn't be an issue since the number of possible properties for each packet is small.

Appending properties is done like this:

    Property<uint32> maxProp(PacketSizeMax, recvBufferSize);

    if (!packet.props.getProperty(PacketSizeMax))
        packet.props.append(&maxProp); // That's possible with a stack property as long as the lifetime of the object outlive the packet

No memory allocation

Unless for very specific cases on your side, no memory allocation is done in the library, and no memory is allocated in the event loop. Everything is pre-allocated upon construction and never evolve. This also means no heap memory fragmentation so your system can run for months without never having to reset due to allocation failure.

Type safe library and no code duplication

C++ template are used wherever possible to limit the library code size. The template usage is very simple (no need to be a C++ guru to understand the template wizardry used in the library). Inheritance is used to avoid template bloat (a great care has be spent to orthogonize the common pattern, each symbol is checked not to share a common byte-code with another one).

Visitor pattern for properties

Unlike MQTT v3.11, in MQTT v5, each packet can store properties to add optional features. Usual MQTT v5.0 clients copy to/from the given value into the packet's property array. eMQTT5 does not, you'll simply use a visitor to get a view on the property. It is both faster and more light on your system resources.

Receiving packets from the broker

When receiving a packet, the code never does any copy, so serialization from Properties is done directly from the received buffer.

In that case, you'll be dealing with PropertiesView class and more specifically with its getProperty method:

bool getProperty(VisitorVariant & visitor) const

Typically, you'll create a VisitorVariant instance then call getProperty. This method will fill each instance with the appropriate visitor and type.

It's then up to you to check which property you are interested in, and extract the visited property value like this:

    VisitorVariant visitor;
    while (packet.props.getProperty(visitor))
    {
        switch (visitor.propertyType())
        {
            case PacketSizeMax:
            {
                auto pod = visitor.as< LittleEndianPODVisitor<uint32> >();
                maxPacketSize = pod->getValue();
                break;
            }
   [...]

When you extract the value from the visitor, you have to specify the type you expect for the given property. In eMQTT5, the possible property type are implementing the standard:

Type Description
PODVisitor<uint8> A memory mapping for a plain old data unsigned 8 bit integer
LittleEndianPODVisitor<T> with T uint16 or uint32 A memory mapping for a plain old data unsigned integer stored as little-endian
MappedVBInt A variable length integer. This has to be converted to the architecture's native int format
DynamicBinDataView A view on a opaque and sized binary array (uint8*)
DynamicStringView A view on a sized char array (int8*)
DynamicStringPairView A view on sized key value char array(int8*, int8*)

Client interface

The client provides everything required for performing the actions above. You'll instantiate a Network::Client::MQTTv5 object and call any of the method below:

MessageReceived interface

MessageReceived interface that must be implemented:

struct MessageReceived
{
    // This is called upon published message reception.
    virtual void messageReceived(const DynamicStringView & topic, 
                                 const DynamicBinDataView & payload, 
                                 const uint16 packetIdentifier, const PropertiesView & properties) = 0;
    // This is usually called upon creation to know what it the maximum packet size you'll support.
    // By default, MQTT allows up to 256MB control packets. 
    // On embedded system, this is very unlikely to be supported. 
    // This implementation does not support streaming so a buffer is created on the heap with this size  
    // upon client construction to store the received control packet. 
    virtual uint32 maxPacketSize() const; // Default to 2048 bytes

    // An authentication packet was received. 
    virtual void authReceived(const ReasonCodes reasonCode, const DynamicStringView & authMethod, const DynamicBinDataView & authData, const PropertiesView & properties) { } // Optionnal: Depends on library configuration
};
MessageReceived::messageReceived
Parameter Usage
topic The topic for this message
payload The (possibly empty) payload for this message
packetIdentifier If non zero, contains the packet identifier. This is usually ignored
properties If any attached to the packet, you'll find the list here.
MessageReceived::authReceived

If enabled in the library configuration, and you are using advanced authentication for your client, this will be called while connecting to the broker with some challenge to succeed. You'll then call MQTTv5::auth method to submit your answer.

By default, no action is done upon authentication packets. It's up to you to implement those packets

Parameter Usage
reasonCode Any of Success, ContinueAuthentication, ReAuthenticate
authMethod The authentication method
authData The authentication data
properties If any attached to the packet, you'll find the list here.

MQTTv5(...)

Signature

MQTTv5(const char * clientID, 
       MessageReceived * callback, 
       const DynamicBinDataView * brokerCert = 0)
Parameter Usage
clientID A pointer to a zero terminated string containing a client ID. Can be nullptr for broker assigned client ID
callback A mandatory pointer to a MessageReceived instance (see below)
brokerCert An optional pointer to a DER encoded certificate of the broker to validate against

Broker certificate

It's not usually possible to store a complete certificate authority chain in an embedded system. And even if it were possible, updating and maintaining such a library would be quite painful. That's why it's possible to store the expected broker's certificate directly and validate against it only.

Using DER for storing a certificate reduce the flash binary size requirement of your certificate by ~33% (compared to plain X509 textual format).

No copy is made from the given pointer, so please make sure the pointed data is valid while this client is valid.

If you don't have a PEM encoded certificate, use this command to save the broker server's certificate to a .PEM file

$ echo | openssl s_client -servername your.server.com -connect your.server.com:8883 2>/dev/null | openssl x509 > cert.pem

If you have a PEM encoded certificate, use this code to convert it to (33% smaller) DER format

$ openssl x509 -in cert.pem -outform der -out cert.der

connectTo(...)

Signature

ErrorType connectTo(
     // Broker configuration
     const char * serverHost, 
     const uint16 port, 
     bool useTLS = false,

     // Session configuration
     const uint16 keepAliveTimeInSec = 300,                                      
     const bool cleanStart = true,

     // Credentials
     const char * userName = nullptr, 
     const DynamicBinDataView * password = nullptr,

     // Last will message
     WillMessage * willMessage = nullptr, 
     const QoSDelivery willQoS = QoSDelivery::AtMostOne, 
     const bool willRetain = false,

     // Additional properties
     Properties * properties = nullptr);
Parameter Usage
serverHost The hostname of the server to connect to
port The 16 bit port of the broker to connect to
useTLS If true, a TLS connection will be attempted. You can specify the expected broker certificate in the MQTTClient constructor
keepAliveTimeInSec The delay before the connection considered lost
cleanStart Should we resume a previous session or clear any previous session
userName The credential's login, empty if using AUTH mode
password The credential's password
willMessage Upon unexpected disconnection, set a last will message here
willQoS The last will message quality of service
willRetain Whether to retain the will message on the topics so it's sent upon client's subscription
properties The properties to embed in the connect packet

Authentication

Authentication in MQTT v3.1.1 was mainly either client identifier based or username/password based, a bit like HTTP/1.0 did. In MQTT v5.0, it's now possible to also have multiple step authentication with per-broker/per-client specific protocol. Authentication requires a new control packet type, and, as such, you can use auth method of the client to build this packet.

The usual process with authentication is the following:

  1. Client => CONNECT with some authentication properties attached => Server
  2. Server => AUTH with some challenge/method/data => Client
  3. Client => AUTH with some answer/data => Server
  4. Server => CONNACK => Client

For the first step, you'll need to append as many properties as required (see the Properties section below for how to do that) in your first call to connectTo .

When the server answers (in step 2), your callback instance will be called with the authentication method and data already parsed for you.

You'll then use the auth method in step 3 to complete the challenge, the method either returns with a success (step 4) or a failure.

Please notice that none of the above is required for usual client identifier or username / password connection.

If enabled in the libary configuration, you'll be able to prepare AUTH packet to send to the broker in reply to a authentication challenge via the auth method:

auth(...)

Signature

ErrorType auth(const ReasonCodes reasonCode, 
               const DynamicStringView & authMethod, 
               const DynamicBinDataView & authData, 
               Properties * properties = nullptr)
Parameter Usage
reasonCode Any of Success, ContinueAuthentication, ReAuthenticate
authMethod The authentication method
authData The authentication data
properties If you need to attach some to the packet, do it here.

subscribe(...)

Signature

ErrorType subscribe(
    const char * topic, 
    const RetainHandling retainHandling = RetainHandling::GetRetainedMessageForNewSubscriptionOnly, 
    const bool withAutoFeedBack = false, 
    const QoSDelivery maxAcceptedQoS = QoSDelivery::ExactlyOne, 
    const bool retainAsPublished = true, 
    Properties * properties = nullptr)
Parameter Usage
topic The topic to subscribe to. This can be a filter in the form a/b/prefix* (prefix can be missing too)
maxAcceptedQoS The maximum accepted quality of service. The client can deal with any level, so you can safely ignore this.
retainAsPublished If true, the Retain flag from the publish message will be kept in the packet send upon subscription (this is typically used for proxy brokers)
retainHandling The retain handling policy (typically, how you want to receive a retained message for the topic)
withAutoFeedback If true, if you publish on this topic, you'll receive your message as well
properties If provided those properties will be sent along the subscribe packet. Allowed properties for subscribe packet are: (Subscription Identifier, User property)

You can call this method as many times as you have topics to subscribe for, or you can use the other subscribe overload with a chained list of topics to subscribe to. When this method returns successfully, MessageReceived::messageReceived callback will be called upon receiving a message.

unsubscribe(...)

Signature

ErrorType unsubscribe(UnsubscribeTopic & topic, 
    Properties * properties = nullptr)
Parameter Usage
topic A chained list of topics to unsubscribe from. Building this packet is similar to what's done in subscribe(const char*... method
properties If provided those properties will be sent along the subscribe packet. Allowed properties for subscribe packet are: (Subscription Identifier, User property)

When this method returns successfully, MessageReceived::messageReceived callback won't be called upon receiving a message on the specified topics. Please notice that's it's required to run the event loop after calling this method to fetch the unsubscribe's result if you are interested in that (which are accessible with getUnsubscribeResult()) method.

It's not necessary to unsubscribe when disconnecting from the broker, so this method is usually non built since a typical application will connect, subscribe then (optionally) publish and disconnect.

publish(...)

Signature

ErrorType publish(
    const char * topic, 
    const uint8 * payload, 
    const uint32 payloadLength, 
    const bool retain = false, 
    const QoSDelivery QoS = QoSDelivery::AtMostOne, 
    const uint16 packetIdentifier = 0, 
    Properties * properties = nullptr)
Parameter Usage
topic The topic to publish to.
payload The payload to send to this publication, can be null
payloadLength The payload length, in bytes
retain If true, the Retain flag will stick to this message and it'll be sent to new subscribers upon subscribing
QoS The quality of service. Any of AtMostOne, AtLeastOne, ExactlyOne
packetIdentifier If non zero, contains the packet identifier. This is usually ignored
properties If provided those properties will be sent along the publish packet. Allowed properties for publish packet are: (Payload Format Indicator, Message Expiry Interval, Topic Alias, Response topic, Correlation Data, Subscription Identifier, User property, Content Type)

eventLoop()

Signature

ErrorType eventLoop()

MQTTv5 has no notion of thread or task. It's up to you to call the event loop at regular interval. The underlying implementation usually fallback to either select or epoll or kqueue so it's safe to call this in a loop with no delay whatsoever. The MessageReceived::messageReceived callback will be called in this loop and any Quality Of Service management will be also performed in this loop.

disconnect(...)

Signature

ErrorType disconnect(const ReasonCodes code, Properties * properties = nullptr)
Parameter Usage
code The disconnection reason.
properties If provided those properties will be sent along the disconnect packet. Allowed properties for publish packet are: (Session Expiry Interval, Reason String, Server Reference, User property)

It's not required to disconnect from the broker in MQTT since there are keep alive and session timeouts. Yet, you can tell the broker for your disconnection reason with such method. After calling this method, you'll have to connectTo again to communicate with the broker.

setDefaultTimeout(...)

Signature

void setDefaultTimeout(const uint32 timeoutMs)
Parameter Usage
timeoutMs The maximum duration to block in millisecond.

By default, when the client isn't configured with LowLatency mode, the eventLoop will block for the some configurable duration, waiting for a packet to be received or some packet to be sent. This is useful to limit the CPU usage since during this wait, the task's timeslice is relinquished for other work on the CPU. However, if you need to perform some other work in the same task as the event loop, this means that you'll see a pauses of the configured timeout between each work.

When built with LowLatency, the eventLoop instantly checks if there is activity on the socket, and if no activity is detected, exits the eventLoop immediately, without blocking. If you have no other work to perform, this will hog the CPU to 100% usage and increase the power consumption of the system. However, if you do have work to perform, and this work throttles itself (for example, by waiting for an interrupt or a timer), this is safe to use.

Previous Post