DesignSpark Electrical Logo
  • Brought to you by
  • RS Components
  • Allied Electronics
peteroakes

November 12, 2016 11:12

Siemens IOT2020 - installing and testing Node-Red, MQTT and SQLITE3

Time to install some software to allow us to build a real open source industrial Gateway

This to me means
Node-Red for control flows to be designed and managed graphically
MQTT for publishing commands and data to upstream systems and also for subscribing and publishing to and from Edge nodes controlling relays and power devices, reading Temperature, Humidity, Pressure... you name it.

SQLITE3 to allow historical data to be stored and retrieved for trending and analysis

Now before we can install any of this you will have had to follow the first few videos
Creating SD Card
Expanding File system to make full use of SD card

 HEALTH WARNING. THIS IS A VERY TECHNICAL POST AND TAKE A WHILE TO COMPLETE THE PROCESS (Over an Hour or more, Making the video while I was at it ment it took me way more). I have tried to document everything correctly but there may be the odd mistake. Please let me know if there are improvements to be made and or corrections.


Now its time to configure Data Sources, Architecture and then install Node-Red, MQTT in the form of Mosquitto and SQLITE3 for database functionality
Then we will add many Node-Red modules allowing Node-Red to use them. We will also add nodes for Modbus, GPIO, Serial and other features.

  1.  Configure repository sources and architecture partameters
    Update Architecture by adding the following lines to /etc/opkg/arch.conf, adding the bolded lines below

    arch all 1

    arch any 6

    arch noarch 11

    arch i586 12

    arch quark 13

    arch x86 14

    arch i586-nlp-32 16

    arch i586-nlp-32-intel-common 21

    arch iot2000 26

  2. add a file called /etc/opkg/iotdk.conf

    insert the following content into the file

    src iotdk-all http://iotdk.intel.com/repos/2.0/iotdk/all
    src iotdk-i586 http://iotdk.intel.com/repos/2.0/iotdk/i586
    src iotdk-quark http://iotdk.intel.com/repos/2.0/iotdk/quark
    src iotdk-x86 http://iotdk.intel.com/repos/2.0/iotdk/x86

  3. run the following for adding a repository for MRAA
    echo "src mraa-upm http://iotdk.intel.com/repos/3.5/intelgalactic/opkg/i586" > /etc/opkg/mraa-upm.conf

  4. run opkg update to update the local lists for repositories

  5. Now time to install nodejs, sqlite3 and a bug fix for sshd. (It gets broken when installing nodejs??), execute the following commands.

    opkg install nodejs

    opkg install sqlite3

    opkg install sshd

  6. Now to install node-red and node modules, execute the following commands (You can copy and past all this in one go to the command prompt, it will work through them automatically)

    npm install -g --unsafe-perm node-red

    cd

    mkdir .node-red

    cd .node-red

    npm install node-red-dashboard

    npm install node-red-contrib-modbus

    npm install mraa

    npm install galileo-io

    npm install node-red-contrib-gpio

    npm install node-red-contrib-upm

    npm install node-red-contrib-modbustcp

    npm install moment

    npm install node-red-admin

    npm install node-red-contrib-bigtimer

    npm install node-red-contrib-esplogin

    npm install node-red-node-sqlite

    npm install node-red-contrib-timeout

    npm install node-red-node-openweathermap

    Now if you dont want all of the modules, then simply remove the line.. This may take a while (Maybe over 1/2 an hour) but it will get there and you may see warnings as well. this is expected with the current version of the OS and these modules. They do however work.


  7. Now we are ready to install Mosquitto, use the following commands, if you have an issue with the wget not finding the file, it may be possible a later version has been released, check repository and update your script to compensate. Again this process can take a while but it will get there.

    cd

    wget http://mosquitto.org/files/source/mosquitto-1.4.10.tar.gz

    tar xzf mosquitto-1.4.10.tar.gz

    cd mosquitto-1.4.10

    # this next step takes a while

    adduser mosquitto

    make WITH_SRV=no

    cd test/broker

    make test

    cd ../../

    cp client/mosquitto_pub /usr/bin

    cp client/mosquitto_sub /usr/bin

    cp lib/libmosquitto.so.1 /usr/lib

    cp src/mosquitto /usr/bin

  8. Setting Node-Red and Mosquitto to auto start on power up or reboot.
    • Node-red, make a file and paste in the following text. File Path:  /etc/init.d/autostart_node_red.sh

      #! /bin/sh

      # for example:

      /usr/bin/node-red -u /home/root/.node-red -v &

      exit 0

    • Mosquitto make a file and paste in the following text. File Path: /etc/init.d/autostart_mosquitto.sh

      #! /bin/sh

      # for example:

      /usr/bin/mosquitto &

      exit 0


    • Now add permissions and set the auto start demon to work

      chmod +x /etc/init.d/autostart_node_red.sh

      update-rc.d autostart_node_red.sh defaults

      chmod +x /etc/init.d//autostart_mosquitto.sh

      update-rc.d autostart_mosquitto.sh defaults

Now we have everything installed we can reboot and see if everything works

 

NOTE for the testing of SQLITE3, the following script can be run to create a data base with tables and set permissions. It will also add a few rows to some tables. If you dont want this, you can simply not use it or just delete it after the test.

cd
mkdir dbs
sqlite3 ~/dbs/iot.db << EOF
CREATE TABLE IF NOT EXISTS \`pinDescription\` (
\`pinID\` INTEGER PRIMARY KEY NOT NULL,
\`pinNumber\` varchar(2) NOT NULL,
\`pinDescription\` varchar(255) NOT NULL
);
CREATE TABLE IF NOT EXISTS \`pinDirection\` (
\`pinID\` INTEGER PRIMARY KEY NOT NULL,
\`pinNumber\` varchar(2) NOT NULL,
\`pinDirection\` varchar(3) NOT NULL
);
CREATE TABLE IF NOT EXISTS \`pinStatus\` (
\`pinID\` INTEGER PRIMARY KEY NOT NULL,
\`pinNumber\` varchar(2) NOT NULL,
\`pinStatus\` varchar(1) NOT NULL
);
CREATE TABLE IF NOT EXISTS \`users\` (
\`userID\` INTEGER PRIMARY KEY NOT NULL,
\`username\` varchar(28) NOT NULL,
\`password\` varchar(64) NOT NULL,
\`salt\` varchar(8) NOT NULL
);
CREATE TABLE IF NOT EXISTS \`device_list\` (
\`device_name\` varchar(80) NOT NULL DEFAULT '',
\`device_description\` varchar(80) DEFAULT NULL,
\`device_attribute\` varchar(80) DEFAULT NULL,
\`logins\` int(11) DEFAULT NULL,
\`creation_date\` datetime DEFAULT NULL,
\`last_update\` datetime DEFAULT NULL,
PRIMARY KEY (\`device_name\`)
);
CREATE TABLE IF NOT EXISTS \`readings\` (
\`location\` varchar(20) NOT NULL,
\`value\` int(11) NOT NULL,
\`logged\` timestamp NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE IF NOT EXISTS \`pins\` (
\`gpio0\` int(11) NOT NULL DEFAULT '0',
\`gpio1\` int(11) NOT NULL DEFAULT '0',
\`gpio2\` int(11) NOT NULL DEFAULT '0',
\`gpio3\` int(11) NOT NULL DEFAULT '0'
);
INSERT INTO PINS VALUES(0,0,0,0);
CREATE TABLE IF NOT EXISTS \`temperature_record\` (
\`device_name\` varchar(64) NOT NULL,
\`rec_num\` INTEGER PRIMARY KEY,
\`temperature\` float NOT NULL,
\`date_time\` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE IF NOT EXISTS \`Device\` (
\`DeviceID\` INTEGER PRIMARY KEY,
\`DeviceName\` TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS \`DeviceData\` (
\`DataID\` INTEGER PRIMARY KEY,
DeviceID INTEGER,
\`DataName\` TEXT, FOREIGN KEY(DeviceID ) REFERENCES Device(DeviceID)
);
CREATE TABLE IF NOT EXISTS \`Data\` (
SequenceID INTEGER PRIMARY KEY,
\`DeviceID\` INTEGER NOT NULL,
\`DataID\` INTEGER NOT NULL,
\`DataValue\` NUMERIC NOT NULL,
\`epoch\` NUMERIC NOT NULL,
\`timestamp\` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP , FOREIGN KEY(DataID, DeviceID ) REFERENCES DeviceData(DAtaID, DeviceID )
);
.exit
EOF

cd
chmod 777 ~/dbs
chmod 666 ~/dbs/iot.db
cd

Now lets configure node-red for some tests.

The MQTT configuration is as follows, your IP address of course may be different, use ifconfig at the linux ssh command prompt to find what it is.

Now the ModBus node, if you dont have a TCP modbus device then you will not be able to test this one, this is also device dependant. I was requesting data from a FLIR AX8 Industrial Thermal Imaging Camera.

The command used for the sqlite3 select statement in node red is this "select * from pins;", the database path is "/home/root/dbs/iot.db"

When you click the button  you should see the debug output as shown

The Dashboard elements are simply entered into and then exited with the done button to set defaults. This will allow them to apear on a web browser. A later video will cover this in more detail.

Browse to the same address as your using now and add /ui at the end, so it will look like this http://192.168.1.178:1880/ui/#/0 but with your IP address.

You should be able to use a browser from any smart device, phone through to desktop. I prefer Chrome but all I tried did work.

The final testing is MQTT, configure MQTT Lense (A Chrome Addon) as shown, adjusting for your IP Address.

I know this is all quite long but necessary. I am hoping that Siemens will add a few of these applications into the base code to make life easier, but at this time its not there and therefor the steps are necessary to get to this point.

If you try this and run into difficulties then please post a command and i'll try to help.

 

peteroakes

I started my career in Industrial Electronics Engineering over 40 years ago, I am also a seasoned Software Architect working as a consultant with large companies and government agencies developing software for Healthcare, Online Reporting solutions and large scale high availability data management. My electronics experience is primarily in automation and control of manufacturing and the gas industry This experience has left me with a good sense of Privacy, Security and reliability from the hardware right through to the data storage in central data centers. My passion has always been electronics. I now make tutorial videos for my youtube channel thebreadboardca. ( http://www.youtube.com/thebreadboardca ). I specialise in Industrial and Home Automation and Control and integration of heterogeneous systems, I love to figure out how to get all sorts of disparate things talking to each other using protocols like Modbus, MQTT etc. Getting a new product, Chip or other item, figuring out what can be done with it and then making a tutorial video is fun to me and I love to share what I learn.

Comments

Dhinesh_siva

June 11, 2017 08:00

Hello Peter,
Now I am using 2.1.3 and location I have installed is /usr/lib/node_modules. In this version too I am facing the same problem the installed nodes are not visible in node-red web page. Kindly suggest me the solution.

Proone

May 1, 2017 09:51

Hi Peter,
I think you were tryharding.
You can download the siemens image here https://cache.industry.siemens.com/dl/files/799/109741799/att_917329/s1/109741799_Example_Image_V2.1.3.zip?download=true .
Where Node-Red is already install and where you can configure serials with the iot200setup command.

Just found this after 2 months working with your tutorial. Better late than never !

luisss749

April 21, 2017 07:25

Hi Peter im in step 5 of this procedure, when i make opkg to node js and i2c-tools im getting some messages about break existing dependencies, is this something serious? how can i fixed?

root@iot2000:~# opkg install nodejs
Not selecting nodejs 0.10.38 as installing it would break existing dependencies.
Package nodejs (4.2.4-r0) installed in root is up to date.
root@iot2000:~# opkg install sqlite3
Package sqlite3 (3:3.11.0-r0) installed in root is up to date.
root@iot2000:~# opkg install i2c-tools
Not selecting i2c-tools 3.1.1 as installing it would break existing dependencies.
Package i2c-tools (3.1.2-r0) installed in root is up to date.
root@iot2000:~# #fix sshd
root@iot2000:~# opkg install sshd
Package openssh-sshd (7.1p2-r0) installed in root is up to date.
root@iot2000:~#

jfourie

March 16, 2017 10:02

Hi Peter. Thanks for all the videos, it does help a lot. I got my IOT2020 last week so I am reading up on all it can do. Can you drive the GPIO pins also with this NodeRed install like I can do on the Raspberry Pi ? I need to switch some relays on and off from a web page.

peteroakes

March 10, 2017 09:59

Dhinesh, What version of the OS are you using. 2.1.1 or 2.1.2
what folder are you in when you run the command to perform the "npm install......"
There is a minor issue with the default node-red on 2.1.2 that is corrected in the soon to be released 2.1.3. I have been working with Siemens to correct a few discrepancies I found. these are all fixable if you know your way arround node-red but can be confusing otherwise.
I will be posting a new video as quickly as I have the completed solution and or when 2.1.3 is released.

peteroakes

March 10, 2017 09:54

anjaliranika, it looks like Intel has adjusted the folder structures of their repository, I will look into this and try to find a solution (Until they do it again :) ). I will post here ones I have an answer

Dhinesh

March 8, 2017 09:52

Hello sir,

Right now I am working with Siemens IOT 2040, I need to connect this device via modbus and datas to transferred to any Cloud network. The problem is that I followed above steps for installing node-modules but in my node-red web page the installed nodes are not visible. I don't know whether it is in problem with node-red version so please guide me for obtaining the solution.

anjaliranika

March 8, 2017 09:56

Hii sir,
When i do opkg update as per the instructions i'm getting following error.. pls help me in solving it out.. i'm new to linux as well as iot2020.. so pls make a response...
i'm pasting the error below:

root@iot2000:/# opkg update
Downloading http://iotdk.intel.com/repos/2.0/iotdk/all/Packages.
wget: bad address 'iotdk.intel.com'
Downloading http://iotdk.intel.com/repos/2.0/iotdk/i586/Packages.
wget: bad address 'iotdk.intel.com'
Downloading http://iotdk.intel.com/repos/2.0/iotdk/quark/Packages.
wget: bad address 'iotdk.intel.com'
Downloading http://iotdk.intel.com/repos/2.0/iotdk/x86/Packages.
wget: bad address 'iotdk.intel.com'
Collected errors:
* opkg_conf_parse_file: Duplicate src declaration (iotdk-all http://iotdk.intel.com/repos/2.0/iotdk/all). Skipping.
* opkg_conf_parse_file: Duplicate src declaration (iotdk-i586 http://iotdk.intel.com/repos/2.0/iotdk/i586). Skipping.
* opkg_conf_parse_file: Duplicate src declaration (iotdk-quark http://iotdk.intel.com/repos/2.0/iotdk/quark). Skipping.
* opkg_conf_parse_file: Duplicate src declaration (iotdk-x86 http://iotdk.intel.com/repos/2.0/iotdk/x86). Skipping.
* opkg_download_backend: Failed to download http://iotdk.intel.com/repos/2.0/iotdk/all/Packages, wget returned 1.
* opkg_download_backend: Failed to download http://iotdk.intel.com/repos/2.0/iotdk/i586/Packages, wget returned 1.
* opkg_download_backend: Failed to download http://iotdk.intel.com/repos/2.0/iotdk/quark/Packages, wget returned 1.
* opkg_download_backend: Failed to download http://iotdk.intel.com/repos/2.0/iotdk/x86/Packages, wget returned 1.

techsage

February 20, 2017 10:43

We automated end to end testing solutions for the testing of MQTT protocol based API's. all the requests: Publisher/Subscriber ... can be automated using a very simple tool of ours. We provide test as a service where the complete API requests are automated using Dynamic test inputs (using your api specifications). with or without Security Authentication token. Our framework is already in use by 3 major Comapnies in energy and IOT Product companies for last 2 years. more information can be found on the following link : http://techsage.eu/iot-taas.html
please mail us for free advise and consultations or a demo of our existing test framework.

johnny66

December 20, 2016 16:57

Hello Peter
A litlle feedback after what im tried....
It seems like when im installing npm it also want to install python-misc package ,but I cannot figure out from where this package src is..
here is terminal output :
******
Installing nodejs (v4.4.3-r1.0) on root.
Configuring nodejs.
root@iot2000:~# opkg install nodejs-npm --force-depends
Installing nodejs-npm (v4.4.3-r1.0) on root.
Installing python-misc (2.7.3-r0.3.0) on root.
Collected errors:
* get_pkg_url: Package python-misc is not available from any configured src.
* opkg_install_pkg: Failed to download python-misc. Perhaps you need to run 'opkg update'?
* opkg_install: Cannot install package nodejs-npm.
*********
I tried to install npm with this command :opkg install nodejs-npm --nodeps
Then npm installs , but I think actually that python-misc are needed because Node-Red would not install..

So that's where I am right now... Hopefully the new image will be out soon :-)

Thanks again for your nice tutorials

Best Regards John

johnny66

December 20, 2016 16:51

Hello Peter
Thanks for your response . I really appreciate it , and I will try this . Anyway its good news that there will be an upgraded image soon.
Best regards John

peteroakes

December 20, 2016 16:51

Hi Johnny66

I have gone back and reviewed the process and found a few changes have occurred since I wrote the post.
1. The repositories have been re-arranged at intel's end, these are now
/etc/opkg/iotdk.conf
src iotdk-all http://iotdk.intel.com/repos/2.0/iotdk/all
src iotdk-i586 http://iotdk.intel.com/repos/2.0/iotdk/i586
src iotdk-quark http://iotdk.intel.com/repos/2.0/iotdk/quark
src iotdk-x86 http://iotdk.intel.com/repos/2.0/iotdk/x86
OR
src iotdk-all http://iotdk.intel.com/repos/3.5/iotdk/galileo/all
src iotdk-i586 http://iotdk.intel.com/repos/3.5/iotdk/galileo/i586
src iotdk-quark http://iotdk.intel.com/repos/3.5/iotdk/galileo/quark
src iotdk-x86 http://iotdk.intel.com/repos/3.5/iotdk/galileo/x86

with the public available image from Siemens, nodejs is already installed and if npm is not there you may need to run "opkg remove --force-remove nodejs" then "opkg install --force-depends node-js" followed by "opkg install force-depends nodejs-npm". I used the 3.5 packages on my image. this resulted in npm being installed correctly,

There will be a new image release by year end but hopefully much sooner. this will include Nodejs, Node-Red etc preinstalled and ready to work saving alot of time after your image is made, it is currently being finished and tested by Siemens, I have seen it and it looks like it will help a lot to simplify getting up and running from a fresh SD image. I have been working with RS and Siemens to get it completed, As soon as it is released I will let everyone know

johnny66

December 20, 2016 16:48

Hello Peter , Thanks a lot for your excelent tutorials ! I am deep down into one of the
IOT 2040 , and are strugglig a little . When I try to install nodejs I get this error :

Not selecting nodejs 0.10.38 as installing it would break existing dependencies.
Package nodejs (4.2.4-r0) installed in root is up to date.

It seems like Node already is installed. Also I cannot use npm as it is not installed

Some Ideas would be really helpfull

Best regards John