Communication - MQTT Sparkplug B testing notes
Last week I tested our MQTT Sparkplug B implementation using the Eclipse™ Sparkplug™ Technology Compatibility Kit (TCK) version 3.0.0. This tool is used by the Eclipse Foundation to test whether an implementation (whether Host Application, Edge Node + Devices, or Broker) meets all the requirements of the Sparkplug 3.0.0 specification. I tested the D2000 KOM process in the role of Edge Node + Devices (data source) as well as the Host Application (data consumer and command sender).
How did it turn out? I fixed a few inconsistencies, one error when converting Sparkplug messages to JSON format (templates) and reported two errors to the developers of the testing tool (one apparently new, the other I later found out was already reported almost two years ago, but it has not been fixed yet).
I would like to stop by specifically when converting Sparkplug messages to JSON format. Ipesoft D2000 allows you to define a text I/O tag with the address IN_SP2JS on a communication station with the MQTT protocol. If a message arrives in binary Sparkplug B format, the D2000 KOM process will convert it to JSON format and publish it as the value of this I/O tag. Of course, in addition to this, standard parsing and publishing to I/O tags with a Sparkplug address still works, as well as publishing the entire message to a text I/O tag with the address IN_DATA.
What is the point of converting the entire message to JSON? Sparkplug B message contains metrics. In addition to the basic attributes that are processed and published in the I/O tag value (name/alias, value type, value itself, timestamp, is_null flag interpreted as Invalid), these also contain others, for example the properties field - in the figure Field(9):
D2000 KOM can convert this metric to JSON format (readable formatting was achieved using the JSON formatter & validator tool, as D2000 KOM does not indent or insert new lines):
The above example uses properties to define limits (engLow=1, engHigh=10) and engineering units (engUnit=My Units).
You may notice that the order of individual attributes of the JSON message is different from the original message (they are sorted alphabetically). The order of individual values (field "values") and names (field "keys") is, on the other hand, the same as in the Sparkplug message, since the order of the array elements matters.
Therefore, if it were necessary to extract information from Sparkplug that the D2000 KOM process does not use by default, this is possible using an I/O tag with the address IN_SP2JS and subsequent parsing of the JSON message in the ESL script, using JSON format manipulation functions.
When testing the message ordering (Message Ordering Test), the D2000 KOM process received a DBIRTH message from TCK, which contains the definition of all metrics of the test server. The image (laboriously assembled from multiple Wireshark snapshots) can be found here, be warned though – it has dimensions of 787 x 12510 pixels.
After converting to JSON (I/O tag with address IN_SP2JS) the result was as follows (text document with JSON message).
Similarly, a smaller NBIRTH message was received. You can see its display in Wireshark (667 x 1027 points) as well as a result of conversion to JSON.
These are of course extremes - BIRTH messages contain all metrics, so they are also reasonably large.
For illustration, I offer the following screenshot from D2000 Cnf, monitored by several I/O tags belonging to a station with the MQTT protocol in the "Sparkplug Host Application" mode. The I/O tag with address IN_SP2JS contains JSON data of a simple message with one metric named Second, which indicates the current second.
In the image, you can see that the JSON also contains an alias item with the value 2. In the Sparkplug B message, a numeric alias is sent as an identifier, and the D2000 KOM process converts it to a text identifier and adds it to the JSON message for the user's convenience (name=Second).
The figure also shows the I/O tag M.MQTT_MOSQUITTO_HA.Second (with address SA=Second), which directly converts the value of the metric Second to an integer value. Such an I/O tag will be sufficient for most use cases.
The I/O tag with address IN_DATA contains the original data in binary Sparkplug format. It is possible to notice how much shorter its value is than the JSON representation and therefore how much more economical the Sparkplug format is (and the use of numerical aliases also contributes to this).
And speaking of aliases and user convenience: if I activate the protocol parameter "Convert Datatype/Timestamp to Text" at the communication line level, then in addition to the datatype and timestamp items, the I/O tag with address IN_SP2JS will have the datatype_txt and timestamp_txt items, indicating the text representation of the data type and timestamp.
This is what the extended JSON message looks like:
For your information, I am also attaching a screenshot of the MQTT Item Browser window, which can be used to configure an I/O tag with an MQTT Sparkplug address. The screenshot shows part of the metrics from the DBIRTH report. You can notice that we can address the entire Dataset column (e.g. using the address SA=DataSet[*]^Booleans). The first row will be published as the I/O tag value, the entire column will be written to the Destination column of the structure (if it is configured in the I/O tag configuration). Of course, it is also possible to address a specific Dataset row (e.g. SA=DataSet[3]^Booleans). Working with template items is also supported (e.g. address SA=TemplateDef->MyFloat).
Conclusion
I am increasingly encountering the MQTT communication protocol and its Sparkplug B extension. A week ago, I configured MQTT Sparkplug B communication in "Edge Node" mode for one of our customers, which is used to publish data from the MES system (built on Ipesoft D2000 technology) to the ThingsBoard visualization platform.
Today, D2000 offers work with text, JSON and Sparkplug B payloads, and for Sparkplug B it also offers its conversion to JSON, which can be useful when processing auxiliary metric attributes (e.g. properties). All this is available to our customers within a single communication protocol.
So be it, the MQTT protocol offers an interesting alternative with its compactness, security support (TLS) and efficiency.
May 28, 2025, Ing. Peter Humaj, www.ipesoft.com