Automating PCB Inspection with Arduino UNO Q Part 2: Development Setup

Automating inspection of printed circuit boards with Arduino UNO Q and Edge Impulse.

In this series of articles, we take a look at how the unique capabilities of Arduino UNO Q (066-5593) may be used to seamlessly integrate AI and control of physical inputs and outputs, with user-friendly training of AI/ML models courtesy of the Edge Impulse software platform.

In Part 1, we outlined the use case and introduced the sophisticated prototyping features provided by UNO Q hardware and the App Lab development environment. In Part 2, we will now take a look at setting up the hardware, getting up and running with Arduino App Lab and Edge Impulse, and preparing PCBAs for model training and testing.

Note that machine learning (ML) is regarded as a subset of artificial intelligence (AI), and this series of articles may use both terms interchangeably. 

Hardware Setup

Our development hardware is comprised of the UNO Q plugged into a USB-C hub, which also has a USB PD power supply, HDMI monitor, USB camera, and a USB wireless keyboard/mouse dongle connected. However, the monitor, keyboard and mouse weren’t really necessary, as pretty much everything can be performed over the network. Though it is a convenient feature that UNO Q can also be used as its own development workstation.

Logitech USB webcam mounted on a tripod.

On first power-up, the UNO Q carries out initial configuration of things such as the WiFi network and user account. This can be driven via either an attached keyboard, monitor and mouse, or using App Lab running on another computer and the UNO Q directly cabled via USB.

The initial setup process will also update the UNO Q — both MCU and MPU parts — to the latest firmware and software. Each time App Lab connects, this will again check for and install any updates, not only for the device firmware and software, but the App Lab IDE itself also.

It’s clear that a lot of effort has gone into the initial setup and update processes, and this makes for a first-class out-of-box experience. Note that should an update ever go awry or flash storage somehow become corrupted, the Arduino Flasher can be used to re-flash the latest or a specific image.

App Lab 101

Arduino App Lab provides an integrated development environment (IDE) for UNO Q, which on Linux is packaged as a convenient single executable, with Windows installers and macOS downloads also available. In addition to which UNO Q comes with App Lab pre-installed for when it is being used as the development workstation.

Upon launching App Lab, it will detect any UNO Q boards that are directly cabled via USB or which are connected via WiFi to the local network. Upon selecting a board, we are then prompted to enter the password, which we earlier set as part of the initial configuration.

We decided to verify our setup by using the Detect Objects on Camera example. Above, we can see the README file for this displayed in the main editor window, with navigation in the left margin and this can be used to open tabs for the Bricks, MCU Sketch libraries and main application files.

The app.yaml file contains metadata such as the application name, icon and description, plus the Bricks used.

If we click on the Video Object Detection brick, we are provided with a description of how this works.

In main.py we can see that the Python module for this brick is imported, along with the module for the Web UI brick. An instance of the UI is created, a video object stream is set up, and a callback is registered for when objects are detected.

We can explore the HTML which provides the user interface for the application.

Along with the JavaScript code, which runs client-side in the web browser, makes a WebSocket connection back to the UNO Q and updates the web page when objects are detected etc.

To launch the application, we simply click on the Run button, with white text output to the console during startup, and this changing to green once the application is fully up and running.

The IP address of the UNO Q can be ascertained from the status display at the bottom of the IDE. Here we can also see how much storage has been used, along with RAM usage and CPU utilisation.

If we SSH to the UNO Q, logging in as the arduino user and the same password as previous, we can execute docker ps and see that two containers have been started, with one bound to port 7000.

A web browser was pointed at the UNO Q IP and port 7000, and the UI loaded. A wireless mouse was placed under the camera and, sure enough, the AI model detected this with a high confidence.

The whole process couldn’t be more straightforward, and if we wanted to, we could now save a copy of an example and start editing this to create our own application. However, next we’re going to take a closer look at Edge Impulse, which simplifies the task of training and deploying AI models.

Edge Impulse Setup

Selecting and training an AI model can be a time-consuming process, as you evaluate the available models and their suitability for the target hardware, figure out the training pipeline, navigate the various parameters, collect and process data, train, test and repeat. Fortunately, Edge Impulse greatly simplifies this task for edge devices and has feature feature-packed free Developer Plan.

An account was created, and we consulted the Edge Impulse documentation for Arduino UNO Q and connected to the board via SSH.

The setup process had us configure the package repository for Node.js and install this along with a number of other dependencies. Next, the Edge Impulse Linux CLI was installed and following which the edge-impulse-linux command was run to connect the UNO Q to Edge Impulse Studio.

Camera Integration

Unfortunately, on first run, this did not detect our Logitech C920 Pro webcam, but a quick search found a solution in the Edge Impulse forum. It turned out that this was due to the plugin used to handle our camera, and we could resolve this by specifying optional GStreamer launch arguments.

Note that it’s likely the Edge Impulse software will be updated so that --gst-launch-args are not required for devices using the libcamera plugin.

Rather than just copying the Gstreamer arguments from the forum post and which were for a different camera, we ran the gst-device-monitor-1.0 command to ascertain the correct string to use.

$ edge-impulse-linux --gst-launch-args "libcamerasrc camera-name='/base/soc\@0/usb\@4ef8800/usb\@4e00000-1.3:1.0-046d:08e5' ! video/x-raw,width=640,height=480 ! videoconvert ! jpegenc"

Command for a Logitech C920 Pro webcam.

The CLI tool then prompted us to enter a device name for the camera.

The command completed successfully, and our UNO Q was connected to the project in Edge Impulse Studio.

A particularly nice feature is that once our device is connected, we can preview the camera and start sampling data directly using the web interface.

A Better Camera

After some experimenting with the Logitech C920 webcam, it became apparent that this would not be suitable for our application, which would benefit from a camera with manual controls and much greater image clarity with small SMD parts.

The webcam was therefore substituted for a Fortex HD5-MIC USB 2.0 Digital Microscope (028-3421) , with features that include a C-mount zoom lens, LED ring illumination, and HDMI, in addition to USB output. This was initially set up with the microscope connected directly to a monitor, for quick and convenient focus and framing. However, so as to avoid having to switch between HDMI and USB output, we installed the guvcview utility on our UNO Q.

$ sudo apt update && sudo apt install guvcview

We could then run the utility locally with a monitor connected to the UNO Q.

Alternatively, if we connected from another Linux computer and used ssh with the -Y option, the graphical output is forwarded back to our local system. Although something to be aware of is that when forwarding an X11 application like this, the framerate may be lower, which is not usually an issue if you are just using this to set the camera up.

A simple fixture to hold our PCBA was laser cut from MDF and positioned so that eight SMD transistors were clearly in view, and then taped in place.

We used v4l2ctl to list the USB microscope supported formats:

$ v4l2-ctl --device=/dev/video0 --list-formats-ext

Which resulted in the following output.

So we had determined that we could stream MJPEG or H.264 encoded video at 30 fps, and with four resolutions to choose from.

We ran into stability issues when using the USB microscope with the Edge Impulse Linux CLI, possibly due to the stream size and our not particularly fast broadband upload speed. However, fear not, as Edge Impulse provide a wealth of options for data acquisition, including using a smartphone to capture images, adding a cloud storage bucket — such as Amazon S3, Google Cloud Storage and Microsoft Azure Blob Storage — and simply uploading images from your computer.

Hence we decided that we’d simply capture stills and manually upload, since the video feed and capture in Studio is just for convenience and ultimately our ML model will be running on the UNO Q. We may at some point come back to this and see if we can construct a Gstreamer pipeline which is more stable, thereby allowing us to preview and sample direct from within Edge Impulse Studio.

$ gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=1 ! image/jpeg, width=640, height=480, framerate=30/1 ! filesink location=PCBA-01.jpg 

The above pipeline configured the camera for 640x480 resolution at 30 frames/second, capturing a single frame to a JPEG file. Of course, the command will be different for different cameras and image formats, but the important thing to note is that we have all the flexibility of Gstreamer and the wider Linux ecosystem when it comes to on-device image capture and formatting.

Another CLI

While the Edge Impulse Linux CLI is for collecting data and running inferences on Linux devices, the Edge Impulse CLI is used to control local devices, act as a proxy to synchronise data for devices that don’t have an internet connection, and to upload and convert local files. In our case this should prove useful for uploading images and save us the additional step of copying them to a laptop and uploading using a web browser.

Since we’d previously installed Node.js, Edge Impulse CLI could be installed on the UNO Q with:

$ sudo npm install -g edge-impulse-cli 

Test Boards

For the purposes of ML model training and testing, we have a fully assembled PCBA, plus a selection of boards where we have removed parts.

Next Steps

Now that we have our hardware set up and verified, tools in place and sample boards, the next steps in part 3 will be to capture images, train an ML model and then test its performance. Following which, we can start to assemble an application which integrates this with physical inputs and outputs.

— Andrew Back