Rounding Up Unruly Zigbees with Zigbee2mqtt
Getting disparate Zigbee home automation systems on a common platform and local control.
Home automation has come on leaps and bounds in recent years, is now commoditised and with many companies offering systems which combine control of heating, lighting, general mains appliances and more. Also featuring remote control via the cloud and mobile applications, along with third-party integrations via their own and partner APIs.
But what if no one system does everything that you require? Well, you could combine multiple different vendor systems if they have APIs, although these are typically cloud-based — i.e. running on remote servers and not some local device — and as you glue these together, the risk of failure increases. Meanwhile, we occasionally hear of companies going out of business and devices being rendered useless or at least with reduced capabilities, as their cloud platforms go offline.
What Zigbee2mqtt does is allow you to take control and create one system to rule them all, where you might have, as we will use in our example, a combined room occupancy, temperature and light sensor from one vendor, together with a combined remote control mains switch and power sensor from another. As the name suggests, these need to use Zigbee wireless and also need to be on the supported devices list — that is unless you’re prepared to work on adding support.
The result is a common platform that, at the time of writing, has out-of-the-box support for 191 Zigbee devices from 39 different vendors. Not only this but a platform that you can run locally and without any dependency on cloud services — unless you decide to integrate them.
Once your devices are accessible via MQTT pub/sub messaging you can then use a home automation system such as Home Assistant or openHAB for the actual control. Or if you really wanted to, you could use Node-RED or perhaps write your own application in any one of the many different languages for which MQTT support is available.
A Raspberry Pi 3 B+ (137-3331) is the perfect platform for running our Zigbee2mqtt gateway and can also easily be used for hosting an MQTT broker and, should we wish, the actual home automation application sitting on top of this. A Zigbee radio is provided by the TI USB 2.4GHz ZigBee Dongle for CC2531 (798-3656). This needs to be programmed with network controller firmware and for this, we used the TI Debugger Programmer for RF System On Chips (709-4370).
The Zigbee2mqtt project provides instructions for flashing the firmware using the TI supplied Windows software, and with the open source cc-tools software on Linux/Mac OS X. We opted for the latter. One thing to note is that the cable adapter board shown on the Zigbee2mqtt website is different to the one we received with the TI programmer, and to get this to work we had to have the connector orientation reversed where it connects to the CC2531 dongle — as shown above.
Having downloaded a firmware image as instructed, it was then simply a matter of flashing this to the CC2531. Now we have our Zigbee controller, we’ll move on to the Zigbee2mqtt software.
What follows is the combined steps for installing Zigbee2mqtt and the Mosquitto MQTT broker, plus command line clients for testing. See also the official documentation.
Assuming that we have a fresh Raspbian install (the Lite version will suffice) and have done any basic configuration if required, e.g. connecting to WiFi, we can start with packaged dependencies.
$ wget -qO - http://repo.mosquitto.org/debian/mosquitto-repo.gpg.key | sudo apt-key add - $ sudo wget http://repo.mosquitto.org/debian/mosquitto-stretch.list -P /etc/apt/sources.list.d/ $ wget -qO - https://deb.nodesource.com/setup_10.x | sudo -E bash - $ sudo apt-get install -y nodejs git make g++ gcc mosquitto mosquitto-clients
Both the Mosquitto MQTT broker and Node.js are available via the Raspbian repositories, but we can get much more up-to-date builds by simply configuring two custom repositories for these.
Next, we clone the Zigbee2mqtt GitHub repository and install.
$ sudo git clone https://github.com/Koenkk/zigbee2mqtt.git /opt/zigbee2mqtt $ sudo chown -R pi:pi /opt/zigbee2mqtt $ cd /opt/zigbee2mqtt $ npm install
We can now run up Zigbee2mqtt with:
$ npm start
To have it started automatically on boot we need to create a systemd unit file:
$ sudo vi /etc/systemd/system/zigbee2mqtt.service
With the contents:
[Unit] Description=zigbee2mqtt After=network.target [Service] ExecStart=/usr/bin/npm start WorkingDirectory=/opt/zigbee2mqtt StandardOutput=inherit StandardError=inherit Restart=always User=pi [Install] WantedBy=multi-user.target
To start via this without rebooting we enter:
$ sudo systemctl start zigbee2mqtt
# Home Assistant integration (MQTT discovery) homeassistant: false # allow new devices to join permit_join: true # MQTT settings mqtt: # MQTT base topic for zigbee2mqtt MQTT messages base_topic: zigbee2mqtt # MQTT server URL server: 'mqtt://localhost' # MQTT server authentication, uncomment if required: # user: my_user # password: my_password # Serial settings serial: # Location of CC2531 USB sniffer port: /dev/ttyACM0
The initial configuration for Zigbee2mqtt can be seen above and to edit this:
$ vi /opt/zigbee2mqtt/data/configuration.yaml
Here you can change the base MQTT topic used, point at a remote host if the broker is running on a different machine, and set a username and password with which to authenticate if this is required.
The serial port should generally be /dev/ttyACM0, but the CC2531 may enumerate to a higher number if you have other devices of that class connected.
Pairing devices and testing
The permit_join parameter should be set to true in order to allow devices to pair. This can later be set to false so as to prevent new devices from joining. Documentation is provided with details of the pairing process for different devices, although we simply powered them up and they joined!
To observe the status of the Zigbee2mqtt bridge and device pairing, we can simply enter:
$ mosquitto_sub -v -h localhost -t zigbee2mqtt/#
Here the # is a wildcard and so we’ve subscribed to all messages under the configured base topic of zigbee2mqtt. Specifying -v means that we’ll get to see what topics messages are being published to. In the below screenshot we can see a Philips Hue motion (occupancy, temperature and light) sensor join and then start publishing messages with the three sensor readings plus battery state. The topic name includes the network address of the device.
We are able to not only receive data via Zigbee2mqtt but also send commands and next we’ll take a look at doing this with a Salus SP600 smart plug.
As soon as the smart plug was powered up it joined and we could start to see reports coming in via the MQTT topic that corresponds to its address. If we then pressed the manual power control button on the plug, the reported state would toggle between on and off.
To set the state remotely via MQTT we could simply use:
$ mosquitto_pub -h localhost -t zigbee2mqtt/0x001e5e0902146174/set -m "ON"
Here we have posted to the same topic as we received reports on, albeit with an added suffix of set and posting a message that contains the desired state.
At this point, we could set up a home automation system that has MQTT support, configure Node-RED or integrate with custom applications. If we wanted, we could make use of one or more cloud platforms, but with the knowledge that we are later free to switch to a different cloud platform or to bring our system fully under local control.
By default, the Mosquitto broker does not require clients to authenticate in order to publish messages and subscribe to topics. This is fine during development and testing, but even if connected to a home network that you consider secure, it’s still a good idea to put some basic authentication in place. How to approach this depends on your overall requirements and you might, for example, simply have clients authenticate to publish, with anonymous subscribe. If you consider data on your MQTT topics to be sensitive, you may require authentication also to just subscribe. Or you may lock down publishing/subscribing to certain topics and users. See the documentation for details.
CommentsAdd a comment
I was rather hoping when I saw the title that it might have something to do with Digi XBees running on Zigbee like the XB24CZ7WITB003 modules.
I have, a network of these devices doing some home energy monitoring using Arduino to coordinate everything. This article seems firmly aimed at commercial devices.
@Phil-S Indeed, that is the case. While I love hacking on fully custom solutions, sometimes it's nice also to be able to take a COTS product that is fully integrated, with a nice compact enclosure and optimised for energy consumption etc. Combined with Zigbee2mqtt this gives you a nice balance of that plus freedom from vendor lock-in and safe in the knowledge that your system will still work if their cloud platform goes offline. Of course, no reason why you cannot also integrate DIY Zigbee nodes into such a system also!