MQTT Persistent sessions allows a client to maintain its session state with a broker even if the connection is lost. This means that if a client disconnects and later reconnects, it will pick up right where it left off, with all of its subscriptions and message queues intact. This can be useful in scenarios where devices may go offline temporarily, such as when they lose power or move out of range of a wireless network. In this blog post, we will discuss the concept of persistent sessions in MQTT and how they can be useful in various IoT scenarios.
When a client connects to an MQTT broker, it establishes a session with the broker. This session is used to keep track of the client’s subscriptions, message queues, and other state information. If the client disconnects from the broker, the session is lost, and the client must re-establish a new session when it reconnects. This can be problematic in certain situations, such as when an IoT device loses power or goes out of range of a wireless network.
MQTT Persistent sessions solve this problem by allowing a client to maintain its session state with a broker even if the connection is lost. This means that if a client disconnects and later reconnects, it will pick up right where it left off, with all of its subscriptions and message queues intact. This can be especially useful in scenarios where devices may go offline temporarily, such as when they lose power or move out of range of a wireless network.
Before moving ahead, lets first understand the concept of QOS in MQTT. MQTT Quality of Service (QoS) level determines the level of guarantee provided for message delivery between the client and the broker. There are three levels of QoS in MQTT: QoS 0, QoS 1, and QoS 2.
To read QoS in details, please click here.
|QoS Level||Delivery Guarantee||Message Sent||Comment|
|QoS 0||Lowest||At most once delivery||This level should be used for messages that are sent over reliable communication links or that can be missed without a problem.|
|QoS 1||Medim||At least once delivery||The message is not considered complete until the sender receives a |
|QoS 2||Highest||Exactly once delivery||This level should be used where the highest level of message delivery reliability is required, and where it is critical that a message is delivered exactly once and not lost or duplicated.|
In relation with MQTT persistent session, MQTT QoS 1 and QoS 2 are used when a device reconnects after a disconnection with a persistent session. The broker will attempt to redeliver any messages that were pending when the client disconnected, up to the QoS level that the device was subscribed with. This way MQTT persistent sessions, combined with QoS 1 and QoS 2, help to ensure that messages are not lost and that the device picks up where it left off, even in case of disconnections.
MQTT persistent session
MQTT Persistent sessions were introduced in MQTT version 3.1.1, which was first defined in the MQTT v3.1.1 specification document.
In MQTT 3.1.1, MQTT persistent sessions are created when the client connects to the broker using a cleanSession flag set to false i.e. 0. This flag tells the broker to maintain the session state, even if the client disconnects. When the client reconnects, it can request the broker to resume the previous session by passing the same client identifier as before and the clean session flag set to false. This way the broker will reconnect the client to the previous session and the client will be able to pick up where it left off.
If no session exists for the client sending the CONNECT message, a new MQTT persistent session will be created. If a session already exists for the client, then the client will resume the existing MQTT persistent session. To clean a session, the client will send CONNECT message using cleanSession flag to 1, and the broker will not store any session state when the client disconnects.
MQTT 5.0 is the latest version of the protocol, and it includes several new features and improvements over MQTT 3.1.1, but the concept of MQTT persistent sessions remains the same. The MQTT 5.0 specification document defines that a client can handle a MQTT persistent session by setting the CleanStart flag and SessionExpiry Interval.
CleanStart flag controls the beginning of the connecting session & the end of the previous MQTT persistent session. When a client set CleanStart to 1 then a new session is created & a previous session is terminated (if any session exists). When the client set the CleanStart flag to false i.e. 0, then the connecting session will resume a previous session, if it exists.
SessionExpiry Interval controls the end of the connecting session. The SessionExpiry Interval specifies the time in seconds, that a session will persist after disconnecting. Setting SessionExpiry interval=0 causes the session to terminate immediately upon disconnect. If the SessionExpiry Interval is not specified in the CONNECT message, the default is 0.
In MQTT 5, CleanStart = 1 & SessionExpiry interval = 0 is the equivalent of an MQTT 3 cleanSession. On otherhand CleanStart = 0 & SessionExpiry interval> 0, is the equivalent of an MQTT 3 persistent session.
Create a MQTT persistent session
A client creates a MQTT persistent session by connecting to the broker using a clean session flag set to “false”. The clean session flag is a Boolean value that is included in the CONNECT packet sent by the client to the broker when it establishes a connection. When the clean session flag is set to “false”, it tells the broker to maintain the session state, even if the client disconnects.
- Here is an example of how a client might create a MQTT persistent session in the MQTT protocol:
- The client sends a CONNECT packet to the broker, including the following fields:
- Client Identifier (client ID) – a unique identifier that the client uses to identify itself to the broker
- Clean Session flag – set to “false” to indicate that the client wants to create a MQTT persistent session
- Will Flag – an optional flag that can be used to specify a last will and testament message to be sent by the broker if the client disconnects unexpectedly.
- The broker receives the CONNECT packet and responds with a CONNACK packet to confirm that the connection has been established. The CONNACK packet includes a return code indicating the status of the connection.
- Once the connection is established, the client can begin sending and receiving messages as usual. If the client disconnects, the broker will maintain the session state, including the client’s subscriptions and message queues.
- When the client reconnects, it sends a CONNECT packet to the broker with the same client ID and the clean session flag set to “false”. This tells the broker that the client wants to resume the previous session.
- The broker receives the CONNECT packet and responds with a CONNACK packet to confirm that the connection has been established. The client is now reconnected to the previous session and can continue sending and receiving messages as if it had never disconnected.
It is worth noting that for a MQTT persistent session, the client ID must be unique across all clients connected to the broker. This is because the client ID is used by the broker to identify the client and its corresponding session state. If a different client uses the same client ID, the broker will assume that it is the same client and will overwrite the previous session state. It’s important for the client to make sure that the client ID is unique and not reused across different clients or connections.
It’s also important to note that the persistence of the session is only guaranteed as long as the broker is running and able to store the session state. If the broker is restarted or shut down, the session state will be lost and the client will have to create a new session when it reconnects.
Additionally, it’s important to consider the resources that the broker has available to store the session state, if too many clients create MQTT persistent sessions, it could cause the broker to run out of resources. So, it’s a good practice for clients to only use MQTT persistent sessions when necessary and clean up the session state when it’s no longer needed.
Join an existing MQTT persistent session
When a client connects to the message broker using a MQTT persistent session, the message broker saves all subscriptions that the client makes during the connection. When the client disconnects, the message broker stores unacknowledged QoS 1 messages and new QoS 1 messages published to topics to which the client is subscribed. Messages are stored according to account limit.
Messages that exceed the limit will be dropped. When the client reconnects to its MQTT persistent session, all subscriptions are reinstated, and all stored messages are sent to the client at a maximum rate of 10 messages per second. In MQTT 5, if an outbound QoS1 with the Message Expiry Interval expires when a client is offline, after the connection resumes, the client won’t receive the expired message.
- A client can join an existing MQTT persistent session by connecting to the broker using the same client ID and the clean session flag set to “false” in the CONNECT packet.
- The broker will look up the session state associated with the client ID and restore it for the client.
- The broker will respond with a CONNACK packet to confirm that the connection has been established and the client has joined the existing session.
End a MQTT persistent session
- The MQTT persistent session expiration time elapses. The MQTT persistent session expiration timer starts when the message broker detects that a client has disconnected, either by the client disconnecting or the connection timing out.
- The client sends a CONNECT message that sets the cleanSession flag to 1.
- A client can end a MQTT persistent session by sending a DISCONNECT packet to the broker, which will tell the broker to close the connection and delete the session state. This will also disconnect the client with broker.
- Alternatively, the broker can also end a MQTT persistent session if the client does not reconnect within a specified time-out period, known as the Keep Alive Interval.
Reconnection after MQTT persistent session expiries
- If a client does not reconnect to the broker within a specified time-out period after ending or losing a MQTT persistent session, the session state will be deleted by the broker.
- When the client reconnects, it will have to create a new session by sending a CONNECT packet with a clean session flag set to “true” or “false” depending on the requirement.
- If the client reconnects with a clean session flag set to “false” and the same client ID, the broker will respond with a CONNACK packet with a return code indicating that the session does not exist.
Best practices about MQTT persistent session
- Use unique client IDs: To ensure that each client has a unique session, assign a unique client ID for each device or application that connects to the broker.
- Keep sessions alive: To prevent a session from expiring, ensure that the client sends a ping message to the broker within the Keep Alive Interval. This will ensure that the session remains active and the broker will not delete the session state.
- Monitor session expiry: Monitor the broker for expired sessions and ensure that clients reconnect before the session expires. If a session expires, the client will have to create a new session and resubscribe to the topic(s).
- Monitor the broker’s resources usage: Keep an eye on the broker’s resources usage when using persistent sessions, as the broker will store the messages and the session information for each client. If the number of clients and/or the frequency of messages increases, the broker’s resources may be overwhelmed, leading to poor performance. Therefore, it is important to monitor the broker’s resources usage and scale the infrastructure accordingly.
- Use of Session Expiry Interval: MQTT 5 introduced the Session Expiry Interval property, which allows the client to specify the duration for which the broker should maintain the session state after the client disconnects. This can be useful for clients that are likely to reconnect within a short period of time.
- Optimize message size: Optimize the size of the messages being sent over the MQTT connection. Large messages can cause increased network traffic and can increase the chances of packet loss, which can affect the performance of the MQTT connection.
- IBM MQTT persistent sessions
- AWS MQTT persistent sessions
- MQTT Version 3.1.1 (oasis-open.org)
- MQTT Version 5.0 (oasis-open.org)