Integrate Bluetooth Low Energy sensor networks with an MQTT to BLE gateway based on Radxa ROCK 3C

This project continues our series setting ROCK boards in an industrial context, showcasing innovative IIoT solutions that can integrate into existing industrial automation, smart agriculture and building management infrastructure or form the building blocks for new solutions.

We build an example gateway capable of connecting Bluetooth Low Energy (BLE) wireless sensors to an MQTT based network. The gateway code supports scanning, connection, service discovery, reading and writing sensor data and notifications using commands encoded into the MQTT message packets, allowing remote control of multiple BLE sensors. The number of gateways can also be scaled to meet demands.

Our chosen hardware is the ROCK 3C (256-3902) due to its powerful RK3566 quad-core processor, on-board Bluetooth support and a Debian CLI image, but you could use any of the boards in the ROCK range, with the addition of a wireless module in some cases.

For the sensor network we are using Silicon Labs Thunderboard development boards. These are packed with different sensors and come pre-programed as BLE Peripherals so they are great for testing and development.

The gateway software has been developed in Python based on best practice guidelines published by the Bluetooth SIG the industry special interests group managing the Bluetooth specification involving leading technology companies.

Full source code is available on the OKdo GitHub: https://github.com/LetsOKdo/bluetooth-gateway-study-guide

The diagram below illustrates how the gateway integrates into our demonstration network, in particular with the MQTT gateway covered in this project: https://www.rs-online.com/designspark/radxa-rock-secure-mqtt-edge-gateway

So for example it would be possible to read remote wireless sensors on the network and use their readings to alter the parameters of Ladder Logic programs running on a PLC over Modbus.

Use cases include environmental monitoring and controlling HVAC appliances in buildings or condition monitoring using BLE sensors in vending machine cabinets. Wireless sensors can be added to existing industrial hardware without compromising existing wiring for example to monitor vibration in a motor and equipment for preventative maintenance and advanced failure warning.

The MQTT protocol enables multiple independent applications to subscribe to the same sensor data via the broker so that data can be collected, stored and processed either locally or by cloud hosted services for AI analytics and record keeping.

Difficulty: Advanced
Time: 4 Hr
Steps: 11
Credit: None
Licence: Eclipse Public License 2.0 - https://github.com/eclipse/mosquitto/blob/master/LICENSE.txt

Parts Needed

Part	Description	RS Stock Number
ROCK 3C	Okdo ROCK 3 Model C 2GB Single Board Computer	(256-3902)
4 x Thunderboard	Silicon Labs BG22 Thunderboard EFR32BG22 Bluetooth Development Kit	(200-9665)
4 x Arduino BLE Sense	Arduino Nano BLE Sense Rev2	(268-6960)
QC Power Supply	OKdo Multihead Dual Port USB Quick Charge Power Supply	(243-6356)

Step 1: Gateway Hardware

You could use any of the ROCK boards for this project, some would require additional wireless modules. I tested it on the ROCK 3C based on the Rockchip RK3566 SoC. This is a 64-bit, quad-core Armv8.2‑A Cortex®‑A55 CPU running at 1.6GHz. The on-board wireless module is compliant with Bluetooth 5.0 and I used the 2MB RAM version. A lower-cost 1 MB version is also available.

The ROCK 3C has a small form factor, great network throughput and more than enough CPU and memory resources for this application. It supports both eMMC and micro SD cards for OS storage and you can add additional storage with an M.2 NVMe or SATA SSD card.

The ROCK 3C fits many of the cases designed for Raspberry Pi 3 Model B+.

Step 2: OS

Radxa supplies a 64-bit Debian CLI image for the ROCK 3C which is built for this purpose so I used that and flashed it to a 16GB SD card. Currently this has a 5.10 kernel, Python 3.9 and the Linux Bluetooth stack, bluez 5.55.

Visit the Radxa Downloads page and grab the latest version of the Debian CLI image for the ROCK 3C:
https://github.com/radxa-build/rock-3c/releases/tag/b43

• Flash the downloaded image using your favourite tool - I use BalenaEtcher.

Before unmounting from your host, open the config partition and edit the file before.txt:

• Update the locale if you’re not in the US

Now you can unmount the SD card, remove it from the host PC and insert it into the ROCK board.

• Optionally attach the USB to Serial cable to the GPIO and open a serial console on your host so you can watch the boot - I use minicom (8/n/1, baud rate 1500000).
• Attach an Ethernet cable to gain remote terminal access over SSH.
• Attach the USB-C power supply and the board should start booting - once it has completed the green LED should be solid and the blue LED will be flashing the heartbeat indicating that the OS is running.
• Log in to the console with rock / rock credentials.

You should be able to ping the ROCK from a Terminal session on your host and login remotely over SSH from the session with password rock:

ping rock-3c
ssh rock@rock-3c

If not login to the board directly and enable and start the ssh service, then try ssh again:

sudo systemctl enable ssh
sudo systemctl start ssh

Step 3: SSH

Before going any further it’s worth locking down SSH access to secure the system.

Change the rock user's default passwords to a secure version:

passwd

Set up SSH for public key authentication and then disable password logins:

• From your host copy the hosts public key to the ROCK’s authorized_keys file using the utility:

ssh-copy-id rock@rock-3c

• Now check you can login without using a password:

ssh rock@rock-3c

• From the remote session on the ROCK create an extension to the ssh daemon config by adding a config file to the sshd_config.d directory with the following contents:

sudo vi /etc/ssh/sshd_config.d/disable_password_logins.conf

ChallengeResponseAuthentication no

PasswordAuthentication no

UsePAM no

PermitRootLogin no

• Now restart the ROCK’s SSH daemon on the ROCK:

sudo systemctl reload ssh

• Now check that password logins fails from your host with Permission denied (publickey), then ssh in again using your public key:

ssh rock@rock-3c -o PubkeyAuthentication=no
ssh root@rock-3c

That’s the system all setup and secured.

Step 4: Thunderboards

Thunderboards are tiny Bluetooth development boards from Silicon Labs based on the EFR32BG22 wireless module. They host a number of on-board sensors that can be controlled over BLE. There’s also an LED, push button, hall effect sensor, IMU and dual mems microphones.

In our examples we will continuously read the Silicon Labs Si7021 relative humidity & temperature sensor.

You can power the boards either using CR2023 coin cells or by attaching them to micro USB cables.

Tip: Heating from the DC/DC voltage regulator affects temperature readings more when connecting with cable.

Out-of-the-box the boards come pre-flashed as a BLE GAP Peripheral device with a GATT Server exposing the on-board sensor characteristics. There’s also a free to use development environment that allows you to program the devices if you want to go down that route. I used them as supplied.

A bit of detective work is needed to obtain the information required to connect and interpret the sensor data. I used the Silicon Labs EFR Connect Mobile App (available from the App Store) to validate the sensor readings and units and the Bluez Bluetoothctl utility on Linux to determine the GATT characteristic handles. See next step.

Testing was performed with 2 x Thunderboard BG22 and 2 x Thunderboard Sense 2 boards (which have additional pressure sensors but are discontinued). You can connect more devices than this, depending on the wireless module in use. The number of gateways can be scaled to accommodate more sensors and different physical locations zones in a network.

The gateway code will work with any BLE device that conforms to the standard.

If you can’t get the Thunderboard’s, Arduino also has BLE enabled boards that are great for this type of project. Try the Arduino Nano BLE Sense (268-6960) but you will have to program this as a BLE device in this case.

Step 5: Bluetoothctl

An initial test makes sure that we can connect and communicate with our Thunderboards using Bluetoothctl. This is a very useful utility which comes as part of the Bluez package and the Debian image recommended has Bluez installed and configured so the rock user has the correct rights to connect Bluetooth devices.

Very briefly, using BLE terminology, to achieve a BLE connection the GAP Central (ROCK 3C) must first scan for Advertising Packets from the GAP Peripheral (Thunderboard). The advertising packet contains the Peripheral’s unique 48-bit MAC Address which the Central can then use to issue a Connection Request to the Peripheral. If the Peripheral accepts the connection, the Peripheral then gives up its GATT Services and Characteristic information to the Central. This information can then be used by the Central to access the sensors.

Tip: If you want a good understanding of Bluetooth LE I recommend reading this excellent resource from the Bluetooth SIG: https://www.bluetooth.com/bluetooth-resources/the-bluetooth-low-energy-primer/

• Power on one of the Thunderboards and the yellow LED in the centre of the board should start blinking, signalling that it is Advertising. This is programmed to only last for 30 seconds to save power.
• From a SSH session on the ROCK start Bluetoothctl:

bluetoothctl

• At the prompt start the scanning for local Bluetooth devices:

scan on

• Make sure the Thunderboard is still advertising (LED flashing). The 48-bit hex address of the Thunderboard should appear in the list of devices discovered in Bluetoothctl. There may be lots of other devices in the area of your scanner that are also advertising and these will appear also.

Tip: put the Thunderboard into advertising mode by pressing the RESET button. If it is connected, this will cause it to disconnect.

• Then, whilst the Thunderboard is still advertising, issue the connect command at the Bluetoothctl prompt using the address in the scan list. You can use tab completion for this, for example one of my devices has the address 84:2E:14:31:C8:B0:

connect 84:2E:14:31:C8:B0

• After a few seconds the Thunderboard should connect (the LED will turn off) and all its services and characteristics will be listed in Bluetoothctl. The Bluetoothctl prompt will also change to include the BLE device’s name when it connects.
• Repeat the process for each Thunderboard until they are all connected.
• You can stop scanning with:

scan off

• After a few seconds, issue the command to list the connected devices which should show your Thunderboards which are connected.
• Exit Bluetoothctl by quitting:

quit

Tip: Type help to get all the commands available in Bluetoothctl and experiment with them.

This screenshot below shows one of my Thunderboards being scanned and connected by Bluetoothctl:

Step 6: GATT Characteristics

The final piece of information we need is the GATT Characteristics Handle that will allow us to read each sensor. This can be seen when each Thunderboard connects using Bluetoothctl.

The bluetooth standard defines UUID’s for a whole load of characteristics so that Central clients can recognise these characteristics automatically.

We can see in the screenshot below that the Thunderboard is using standard values for Temperature and Humidity as they are identified automatically when the board connects. The unidentified Vendor specific characteristics would need further investigation to determine what they relate to. These are configured in the firmware supplied with the Thunderboard before it was compiled.

The Bluetooth standard 128-bit UUID for Temperature is:

00002a6e-0000-1000-8000-00805f9b34fb

And from the bluetoothctl output, the device handle on this particular device is:

/org/bluez/hci0/dev_58_8E_81_A5_4B_10/service001f/char0022

Collect all the device handles and copy them somewhere safe as we will use them later.

Step 7: Python Paho

Eclipse paho-mqtt is an open source client package for Python that makes it easy to adds support for communicating with MQTT brokers to your Python applications: https://pypi.org/project/paho-mqtt/

To get the latest version we will install it into a Python virtual environment from PyPi, the Python package repository.

In a new terminal session execute the following to setup your locale and install the package in a virtual environment:

sudo dpkg-reconfigure locales
sudo apt update
sudo apt upgrade
sudo apt install python3-venv
mkdir -p ~/python && cd ~/python 
python -m venv venv
. venv/bin/activate
pip install paho-mqtt

We need to make the dbus module from the system packages available to the virtual environment. To do this edit venv/pyvenv.cfg and set include-system-site-packages = true

Then deactivate the environment and reactivate it again to load the new settings:

deactivate
. venv/bin/activate

That’s it, the GAP / GATT modules will be provided by the gateway installation next.

Step 8: Gateway Installation

The Bluetooth SIG (Special Interest Group) has produced a guideline for BLE developers containing a working implementation of a BLE gateway over TCP/IP, along with good documentation explaining how it all works. It also includes a Python package providing modules which support the GAP & GATT protocol for communicating with BLE devices.

The guideline can be downloaded here: https://www.bluetooth.com/bluetooth-resources/bluetooth-internet-gateways/

We will add new Python modules to enable MQTT protocol and translate this into BLE protocol using an adapted version of the GAP and GATT packages contained in a fork of the repository.

The full source code is available on the OKdo GitHub along with the original Bluetooth SIG code, which is redistributable under their EULA:

• Clone the gateway repo to your host PC:

git clone https://github.com/LetsOKdo/bluetooth-gateway-study-guide.git

• From your HOST copy just the gateway directory to the ROCK users home dir:

cd 
python/bluetooth-gateway-study-guide/implementation/solutions/peripherals/
scp -r gateway_mqtt_no_security/ rock@rock-3c:~/python/

• This will install the MQTT / BLE gateway code into the following location on the ROCK:

/home/rock/python/gateway_mqtt_no_security/

Step 9: Gateway Testing

To ease testing, the Mosquitto broker and client packages can be installed onto the gateway.

• From an SSH session on your Host:

ssh rock@rock-3c
sudo apt install mosquitto mosquitto-clients

• For testing purposes, only unsecured connections are used so the broker will need to be configured to accept anonymous connections on port 1883. Edit the configuration:

sudo vi /etc/mosquitto/conf.d/broker.conf

• Enable anonymous access on the default MQTT port 1883 by adding the following settings:

# Enable settings by listener
per_listener_settings true
# Allow anonymous access on port 1883
listener 1883
allow_anonymous true

• Restart mosquitto and check that is is running:

sudo systemctl restart mosquitto.service
sudo systemctl status mosquitto

I have written a Paho MQTT client that implements all the functions necessary to scan, connect, discover BLE services and Characteristics and Read and Write to the sensors on the Thunderboard. So to be able to automate the commands that we applied manually using Bluetoothctl.

Commands are sent to the client in the form of MQTT messages, with the command embedded in the message payload. The client is an MQTT Subscriber that listens for the incoming MQTT messages from the broker, extracts the BLE command and dispatches it to the correct Thunderboard device.

For everything to work, commands must be sent in the correct sequence from another Publisher:

1. Turn on scanning and discover devices that are advertising
2. Connect to device using its address
3. Discover services using the device address
4. Read or write to the device characteristic (sensor)
5. Enable notifications on characteristics so that the Central receives updates automatically

To start the client open a separate SSH session, change to the client directory and run the module, passing in the hostname of the MQTT server, which in this case is localhost as everything is running on the ROCK-3C and the MQTT topic name, you can chose this, I used test/gateway.

cd python/gateway_mqtt_no_security/gateway
python -m mqtt_client “localhost” “test/gateway”

Here you can see the client has started and is waiting for inbound messages from the broker:

Tip: I have copied example MQTT message commands into the Docstring of mqtt_client.py. These will need adapting for the address and characteristic handles for your own Thunderboards.

Assuming that your Thunderboards are still connected using Bluetoothctl, on the host PC open another terminal session and SSH into the ROCK.

ssh rock@rock-3c

Copy and paste a mosquitto command to turn on the Thunderboards LED. For example on my Thunderboard with address 58:8E:81:A5:4B:10 this would be:

mosquitto_pub -h localhost -t "test/gateway/in/write_characteristic" -m '{"bdaddr":"58:8E:81:A5:4B:10", "handle":"/org/bluez/hci0/dev_58_8E_81_A5_4B_10/service002e/char0034", "value":"01"}'

Here I am using the mosquitto_pub client to publish a message to the broker running on host localhost with a topic path test/gateway/in/write_characteristic. The message body contains the command encoded as a json string, containing the MAC address of the Thunderboard, the characteristic handle and the command value as an 8-bit HEX integer as a string. “01” turns the LED on and “00” off.

The LED will turn on and you should see the MQTT command details and the response in the MQTT client terminal. A result code of 0 indicates a successful command response. The codes are in the bluetooth.bluetooth_constants.py module.

If your output looks similar to this, congratulations, your gateway is working!

Step 10: Reading sensors

As an example of reading sensors remotely, remote_client.py issues read commands to the various environmental sensors on the Thunderboard and waits asynchronously for the response sent over MQTT, then displays the results. This client also converts and scales the returned values to human readable format.

remote_client.py has a hard coded dictionary structure containing the commands for reading each of the sensors on the Thunderboard. These will need editing for your devices.

In a production system they would be configured and read in externally from a file or data store.

# Dictionary of command strings to read characteristics from BLE sensors
sensors = {
    "temperature_sensors": [
        '{"bdaddr":"90:FD:9F:7B:7E:E0", "handle":"/org/bluez/hci0/dev_90_FD_9F_7B_7E_E0/service001b/char0020"}',
…
    ],
    "pressure_sensors": [
        '{"bdaddr":"90:FD:9F:7B:7E:E0", "handle":"/org/bluez/hci0/dev_90_FD_9F_7B_7E_E0/service001b/char001e"}',
…
    ],
    "humidity_sensors": [
        '{"bdaddr":"90:FD:9F:7B:7E:E0", "handle":"/org/bluez/hci0/dev_90_FD_9F_7B_7E_E0/service001b/char0022"}',
…
    ],
}

The commands are sent to the gateway continuously in a loop and the returned values are then published by the command objects. For temperature readings, these are 16-bit byte arrays encoded as Hex strings (big endian) which are scaled and converted into human-readable format by the callback function in remote_client.

• Make sure that the mqtt_client is started and the Thunderboards are connected.
• Start the remote client from another SSH session on the ROCK and attach it to localhost with the same topic route as mqtt_client:

ssh rock@rock-3c
cd python/gateway_mqtt_no_security/gateway
python -m remote_client "localhost" "test/gateway"

In the MQTT client session, you should see the MQTT commands being received from the remote client; showing the device MAC, characteristic handle, command, 16-bit Hex value (big-endian) returned and command result code:

In the remote client session, you will see the responses being published by the sensor showing; MQTT topic, device MAC and formatted sensor type and value.

The values are converted from HEX format into scaled, decimal values for each type of sensor by the remote client. If the characteristic used is one of the standard BLE characteristics from the specification, then the spec defines this conversion so remote clients can perform it correctly. If not then the conversion must be done according to the specification of the application running on the BLE peripheral, or with detective work in our case with the Thunderboard mobile app.

Step 11: Troubleshooting

If you are having issues there are a few things you can check to narrow things down:

• Python will give a traceback error if there’s a problem. The code currently has very little in the way of error handling to keep it as simple as possible. Read the error traceback carefully as it will have clues as to where the problem originates.
• Check that the Mosquitto broker is running, the mosquitto service must be active and running without errors:

systemctl status mosquitto.service

• Check that the parameters (hostname & topic root) used by both mqtt_client and remote_client match.
• Check the present working directory in each session is set to (if you installed the gateway to the rock users home directory):

rock@rock-3c:~/python/gateway_mqtt_no_security/gateway$ pwd
/home/rock/python/gateway_mqtt_no_security/gateway

• Check the Thunderboards are actually connected using Bluetoothctl in a separate remote session on the ROCK:

rock@rock-3c:~$ bluetoothctl 
Agent registered
[CHG] Controller 50:5A:65:27:45:D8 Pairable: yes
[Thunderboard #19216]# devices
Device 84:2E:14:31:C8:B0 Thunderboard #51376
Device 58:8E:81:A5:4B:10 Thunderboard #19216
…

Summary

In this project we implemented a working MQTT to BLE gateway that runs on ROCK boards and can communicate with 4 or more BLE devices, in this case Silicon Labs Thunderboards. The Python clients will work with any standard BLE devices.

MQTT messages containing BLE commands are issued by the Central (ROCK) to the BLE Peripheral (Thunderboard) devices connected to the gateway, allowing integration of a network of BLE sensors into an MQTT based network.

The gateway code is based on guidelines published by the Bluetooth SIG who maintain the Bluetooth specification. The full guideline and documentation is published on the OKdo GitHub and is redistributable under a EULA.

We have shown how to prepare, install and run the Python code that executes on the gateway. We illustrated using bluetoothctl to test connecting BLE devices to the gateway and a remote client script. A remote client that reads BLE sensors attached to the gateway using MQTT protocol was also tested.

Although the project was implemented on a ROCK 3C it will run on any ROCK board with Bluetooth hardware. It was designed as part of a larger MQTT based network but can also be used standalone in your own MQTT network projects.

References:

• Project source code
• Bluetooth LE Primer
• Bluetooth Gateways
• Bluetooth specifications
• Eclipse Paho
• Mosquitto project
• Build a secure Mosquitto MQTT gateway