MQTT manages devices, collects data

2We are seeing more and more intersection between enterprise networks and embedded data gathered from many sources. The MQTT protocol has gained notoriety as a choice embedded and enterprise teams can agree on to exchange critical data quickly, reliably, and easily.

Ten years ago, IBM’s Andy Stanford-Clark collaborated with Eurotech’s Arlen Nipper to author the first version of the Message Queuing Telemetry Transport (MQTT) protocol. Back then, Andy noticed IBM’s customers were getting data from remote locations and then using the data in their businesses.

The goal was to extend the reach of enterprise messaging applications through remote data-generating devices. In addition, the technology needed to work in various industries, such as oil and gas, health care, transportation, and any business with remote data collection. Bandwidth was expensive, especially over satellite networks.

Photo | Eurotech’s Arlen Nipper (left) and IBM’s Andy Stanford-Clark (right) recount the development of MQTT at a 10th anniversary party for the protocol in July 2009. Photo courtesy of Tony Whitmore.

The group quickly found there were no standards in this area, so they developed a generic messaging protocol that everyone could use to solve the problem of interoperability. The result was MQTT, a lower-cost, concise messaging protocol optimized for resource-constrained devices.

Not just HTTP or SOAP

As described at www.mqtt.org, the MQTT protocol enables a publish-and-subscribe messaging model in an extremely lightweight way. It is useful for connections with remote locations where a small code footprint is required and/or network bandwidth is at a premium.

Eurotech has used MQTT for device management and data collection since its inception, and the company aims to share this openly published protocol with peers to introduce it as an alternative to HTTP and SOAP for device communication.

Many developers are entering embedded computing with an IT background. Because there is no true industry standard in data communications protocols, they gravitate toward the protocols they have used before: HTTP and SOAP. Some of the drawbacks associated with these options include thousands of bytes sent as a header for the message, rigid formats, and point-to-point communication. The MQTT protocol, on the other hand, stands above the rest for three reasons: it is a lightweight protocol to save on bandwidth; it offers three Quality of Service (QoS) levels; and it enables a publish/subscribe model for efficient one-to-many communications.

Lightweight protocol

MQTT sends and receives data in the form of messages to and from applications and devices over TCP/IP network connections. The protocol is optimized for low-bandwidth communications networks with brief messaging headers to conserve bandwidth.

MQTT utilizes many advantages of TCP that some software developers ignore. Once the TCP socket is established, a few bytes can be sent alone without any overhead. By sending less information back and forth when communicating between devices, you generate far less traffic compared with other protocols. MQTT also utilizes the TCP level to acknowledge the receipt of information, rather than requiring an acknowledgement at a higher level.

In addition, MQTT is data agnostic. The protocol allows linking any kind of sensor with any kind of enterprise application, so there is no worry about how the data gets from point A to point B. Because any type of data can be sent, MQTT has the flexibility to support many types of data and devices.

QoS

The ability to guarantee the delivery of information is imperative to distributed data collection and device management. With MQTT, a maximum QoS can be specified in a subscription message. The three levels of QoS are:

  • At most once: (Also called fire and forget.) Data is sent once, with no resend. Acknowledgement is not expected.
  • At least once: Data is sent once until delivery is assured and an acknowledgement of receipt is sent. The data may be received multiple times due to retransmission.
  • Once and only once: Data is sent once and assured not to be duplicated.

Other protocols simply drop messages if they are not delivered. The QoS features of MQTT increase reliability and allow the message author to specify which level is preferred.

To provide an example of how the QoS levels are used, consider an oil well that has burst and is leaking oil. The oil company needs to clear the error (Editor’s note: and we really, really would like it to) and does not care how many times the message is sent until the error is cleared. In this case, the at least once QoS is used. A bank transaction, on the other hand, would use the once and only once QoS. To ensure an ATM transaction is not deducted from an account twice, the message is only sent once.

Publish/subscribe

MQTT provides publish-and-subscribe messaging (shown in Figure 1) that enables devices to send and receive alerts and data when significant events occur.

With a single publisher and many subscribers, engineers can send information from a single point to many other devices or listeners interested in receiving the information. This concept is similar to Twitter in that a single person posts information and many subscribers view the information simultaneously.

MQTT for distributed data collection in a warehouse

To understand how to use MQTT in the real world, consider a retail warehouse with many devices gathering incoming and outgoing product information. Older IT systems would require employees with barcode scanners to scan boxes going in and out to monitor inventory. The information is then stored using an IT infrastructure with a back end to analyze and track data.

Data collected by the barcode scanners must be input back into the IT infrastructure and sent to a main office so that managers can analyze what came in and out of the warehouse and use it to run the retail business.

A more streamlined approach would be to utilize RFID antenna arrays with an edge controller. Data can enter the IT infrastructure through the antenna and edge controller, and then the edge controller can publish it to the cloud or to the retailer’s back end via a protocol like MQTT. If the retailer does not want to build its own network in the warehouse, this approach coupled with a cellular modem can do the trick at a lower cost, while allowing the company to focus on its core competencies instead of maintaining an IT infrastructure.

This retail warehouse example demonstrates the advantage of the publish/subscribe model. When a box comes in from a supplier, the data can be published to multiple subscribers, including the retailer, shipping company, and supplier. Thus, all three parties know when the box is delivered successfully.

MQTT for a connected world

Eurotech has developed and supplied embedded systems with MQTT drivers for years. The company’s Everyware Software Framework enterprise bundle includes MQTT to allow OEMs to quickly transmit data using this lightweight protocol.

MQTT was developed to connect businesses to data, no matter how remote the location. Embedded devices can utilize the MQTT protocol to collect data from multiple devices while using limited bandwidth and providing the information to many subscribers. As a result, the system is relatively simple to set up and manage for distributed data collection. IES

Wes Johnson is a senior software engineer at Eurotech Inc. responsible for application software and application frameworks on ARM and x86 SBCs. He has been with Eurotech for six years working primarily in a Linux environment with a focus on Java, C, and shell scripting development. Wes received a BS in Computer Engineering and a BS in Physics from Portland State University.

Eurotech 301-490-4007 www.eurotech-inc.com