MQTT: Complete Definition and Guide

What is MQTT?

MQTT, which stands for Message Queuing Telemetry Transport, is a lightweight network messaging protocol specifically designed for machine-to-machine (M2M) communications and the Internet of Things. Created in 1999 by Andy Stanford-Clark (IBM) and Arlen Nipper to monitor oil pipelines over expensive satellite links, MQTT was designed from the outset to operate with minimal bandwidth and processor resources. It became an OASIS standard in 2014 and an ISO standard (ISO/IEC 20922) in 2016.

The fundamental principle of MQTT is based on the publish/subscribe (pub/sub) model, which differs radically from the classic client/server model used by HTTP. In this model, devices do not communicate directly with each other. They interact through an intermediary called a broker (message broker). A device that wants to send data "publishes" it on a "topic", while devices interested in that data "subscribe" to that topic. The broker handles routing messages from publishers to the appropriate subscribers.

This decoupled architecture offers considerable advantages in terms of scalability, flexibility, and reliability. Adding a new sensor or data consumer requires no modification of existing components. The MQTT protocol uses TCP/IP as its transport layer, ensuring ordered packet delivery, and its lightweight nature makes it perfectly suited to resource-constrained microcontrollers such as the Raspberry Pi, ESP32, or Arduino.

Why MQTT matters

In the IoT ecosystem, the choice of communication protocol is a fundamental architectural decision that impacts the performance, reliability, and scalability of the entire system. MQTT has established itself as the reference protocol for several reasons.

• Minimal footprint: an MQTT message header is only 2 bytes, making it ideal for low-bandwidth networks (satellite, LoRaWAN, expensive cellular networks) and memory-constrained devices.
• Complete decoupling: the pub/sub model decouples data producers and consumers in space (no need to know IP addresses), in time (no need to be connected simultaneously), and in synchronisation (no blocking during operations).
• Configurable Quality of Service (QoS): MQTT offers three QoS levels (0: at most once, 1: at least once, 2: exactly once), allowing delivery reliability to be adapted to each data type's needs.
• Retained messages and testament: the broker can store the last message published on a topic (retained message) and automatically notify subscribers when a device disconnects unexpectedly (Last Will and Testament).
• Horizontal scalability: a single MQTT broker can handle hundreds of thousands of simultaneous connections, and brokers can be clustered for very large-scale deployments.
• Massive adoption: MQTT is supported by all major cloud providers (AWS IoT Core, Azure IoT Hub, Google Cloud IoT), ensuring interoperability and investment sustainability.

How it works

The MQTT architecture revolves around three components: clients (publishers and subscribers), the broker, and topics. The communication flow proceeds in several steps.

An MQTT client first establishes a connection with the broker by sending a CONNECT packet containing a unique identifier, optional credentials (username/password), and session parameters (clean session, keep-alive interval, Last Will). The broker responds with a CONNACK packet confirming or refusing the connection.

Once connected, a client can publish messages on a topic. Topics are organised hierarchically with levels separated by forward slashes, for example: building/floor-3/temperature-sensor/value. This hierarchy allows subscribers to use wildcards: + replaces a single level (building/+/temperature-sensor/value captures all floors) and # replaces all remaining levels (building/# captures everything about the building).

The Quality of Service (QoS) determines the delivery guarantee. QoS level 0 (fire and forget) sends the message once without confirmation, ideal for frequent telemetry data where occasional message loss is acceptable. QoS 1 guarantees delivery at least once through an acknowledgement mechanism (PUBACK), but duplicates are possible. QoS 2 guarantees delivery exactly once via a four-step handshake (PUBREC, PUBREL, PUBCOMP), suited for critical commands such as actuator activation.

Security is ensured through TLS/SSL encryption for transport, authentication via credentials or X.509 certificates, and access control lists (ACLs) to restrict publication and subscription rights per topic.

Concrete example

In Kern-IT's IoT projects, MQTT forms the backbone of communication between field devices and our backend. Consider the example of a monitoring system deployed for a telecom operator. Each Raspberry Pi installed in a technical cabinet runs a Python MQTT client (Paho library) that publishes sensor readings every 30 seconds on structured topics: kern/telecom/site-{id}/temperature, kern/telecom/site-{id}/humidity, kern/telecom/site-{id}/vibration.

On the server side, our Django application integrates an MQTT subscriber that listens to the root topic kern/telecom/# using the multi-level wildcard. Received messages are deserialised, validated, and stored in a PostgreSQL database. An alerting system checks in real time whether values exceed configured thresholds and, if so, sends notifications via WebSocket to the dashboard and by email to the relevant technicians.

The geolocated data is then visualised on KERN MAP, our mapping platform based on PostGIS and Leaflet, where each site appears with a colour code reflecting its status (green, amber, red). MQTT's Last Will and Testament feature allows us to automatically detect if a Raspberry Pi disconnects unexpectedly, triggering a "site offline" alert on the map.

Implementation

1. Choose an MQTT broker: for medium-sized projects, Mosquitto (open source, lightweight) is perfectly suitable. For large-scale deployments, consider EMQX or HiveMQ, which offer clustering and enterprise features.
2. Define the topic hierarchy: design a clear, extensible naming convention, for example {organisation}/{project}/{site}/{sensor}/{measurement}. Document this convention for the entire team.
3. Configure security: enable TLS to encrypt communications, configure authentication (credentials or client certificates), and define ACLs for each client type.
4. Implement clients: on IoT devices (Raspberry Pi), use the Paho MQTT library in Python. Configure the appropriate QoS for each message type (QoS 0 for frequent telemetry, QoS 1 for alerts).
5. Develop the backend subscriber: integrate an MQTT client into your Django or FastAPI application that listens to relevant topics, processes messages, and persists them to the database.
6. Monitor the broker: track broker metrics (active connections, messages per second, latency) to detect performance issues before they impact the service.
7. Test resilience: simulate network disconnections, load spikes, and broker failures to validate system behaviour under degraded conditions.

Associated technologies and tools

• Eclipse Mosquitto: lightweight open-source MQTT broker, ideal for deployments on Raspberry Pi or in Docker containers.
• Paho MQTT: MQTT client libraries available in Python, JavaScript, Java, C, and Go, maintained by the Eclipse Foundation.
• EMQX / HiveMQ: enterprise MQTT brokers with clustering, bridging, and web management interface.
• MQTT Explorer: graphical tool for visualising and debugging MQTT topics and messages during development.
• Node-RED: visual flow programming tool, often used for rapid prototyping of IoT flows with MQTT.
• InfluxDB / TimescaleDB: time-series databases optimised for storing sensor data received via MQTT.
• WebSocket: MQTT can operate over WebSocket, allowing web browsers to subscribe directly to topics for real-time updates.

Conclusion

MQTT is the essential protocol for any IoT solution aiming for reliability, scalability, and bandwidth efficiency. Its conceptual simplicity conceals remarkable power, and its industry adoption makes it a sustainable choice. At Kern-IT, MQTT is at the heart of our IoT architecture: it connects the Raspberry Pi units deployed in the field to our Django backend and feeds real-time data to our KERN MAP mapping platform. If you are considering an IoT project, MQTT should be your first choice for a communication protocol.