Jetson AGX Orin and Orin Nano Comparison Part 2: Performance

A side-by-side comparison of the flagship and entry-level developer kits in NVIDIA’s latest and greatest family of embedded platforms for edge AI.

In previous articles we’ve taken a look at the original Jetson Nano platform (basic hands-on and AI follow trolley project), and the powerhouse that is the Jetson AGX Orin (first look and basic setup).

In this two-part series, we compare and contrast the developer kit for the top-end Jetson AGX Orin, with the smaller and much lower-cost Jetson Orin Nano. We started in Part 1 by exploring the features and specifications of each. In the second and concluding article, we will now execute some example workloads and compare the relative performance of the two platforms.

Orin Nano setup

We previously covered basic setup for the AGX Orin  (253-9662) and thought it worth saying a few words about Orin Nano  (264-7384) setup. Whereas with AGX Orin there is a soldered-down eMMC flash with a pre-installed image, the Orin Nano is supplied without any persistent storage and so it’s necessary to provide this. E.g. a Micro SD card or NVMe SSD.

An image is provided which can be written out to an SD card, but when we did this we found that the Orin Nano did not properly boot; the NVIDIA logo was displayed and a message about attempting direct boot, following which the display went blank and the fan stopped spinning. Searching the forums found posts where it was suggested to use the SDK Manager method.

Using SDK Manager also meant that we could try out a custom install and target an NVMe SSD. For a walkthrough of this process, see the article, NVIDIA Jetson Setup Using SDK Manager.

A Level Playing Field

Bearing in mind that the AGX Orin board was set up via an image which had been factory pre-installed to the eMMC, while the Orin Nano was configured more recently using SDK Manager, it was likely that they were running different software releases. Although you wouldn’t expect this to make a huge difference if the releases are not too far apart, it’s only fair to have both running as close as possible to the same software environment when making performance comparisons.

Upon executing the following command on the Orin Nano:

$ cat /etc/nv_tegra_release

We received the output pictured above, indicating that we were running Jetson Linux 35.4.1.

Executing the same command on the AGX Orin system we received the above output, indicating that this dev kit was running Jetson Linux 35.2.1, a slightly older release.

Comparing the file /etc/apt/sources.list.d/nvidia-l4t-apt-source.list on both systems, we could see that the package repository on the Orin Nano was configured for release r35.4. Whereas when we examined the same file on the AGX Orin system we could see it was configured for release r35.2.

To upgrade the Orin AGX system to the same level as the Orin Nano, we updated the release specified in the aforementioned file, then executed an APT update and dist-upgrade, followed reboot.

$ sudo apt update && sudo apt dist-upgrade && sudo reboot

Packages were subsequently upgraded and this appeared to complete without error.

Upon rebooting, the console display then indicated that further updates were being carried out.

Running the command again on the AGX Orin we could see it had now been updated to Jetson Linux 35.4.1.

At this point, we noticed that the developer documentation warned that you should not run an “OTA update” such as this if you have JetPack installed, and to consult its documentation. Following which it was confirmed that JetPack was indeed installed on the AGX Orin, but not the Orin Nano.

However, upon consulting the JetPack upgrade documentation, it seemed that there was just one additional step to carry out, to fix broken packages — and running this appeared to have no effect.

Since we had been alerted to the fact that the Orin Nano did not have JetPack, this was installed with:

$ sudo apt install nvidia-jetpack

At this point, we should now have the same software environment on both development kits.

Quick Test

For a quick first check to make sure that we can access the GPU via CUDA and to report on some headline capabilities, we can build and run the deviceQuery example which we covered previously in the article, NVIDIA Jetson AGX Orin Basic Setup. Fragment of Orin Nano output shown above.

Here we can see the output when we run the same example on the AGX Orin. Thereby confirming that we can access the CUDA device (GPU) on both systems and that they have the same driver and runtime versions. We can also see how the AGX Orin has a lot more global memory and the GPU max clock rate is almost double. However, this by no means shows us the full extent of the differences and AGX Orin has a number of other advantages over Orin Nano, such as a total of 2048 CUDA cores (the figure shown here is different) and 64 instead of 32 Tensor cores.

AI Examples

We’ll be running three inference examples on each Jetson dev kit, processing video from an attached USB camera, and then comparing the results.

The examples we’ll be using are from Hello AI World:

1. Camera segmentation multi-human parsing (MHP) using the ResNet-18 network.
2. Background removal with the U2-Net network.
3. Camera detection with the MobileNetSSDv2 model.

The first time an example is run there will be a delay before the application starts, as the model is downloaded, set up and tuned. This is most notably the case with the background removal example, which takes a little while to complete with either of the Jetson Orin dev kits.

In these tests, we’re using a live camera feed and looking at a rough average performance comparison. A more scientific test might be to record video clips which are used in each comparison and to modify the examples to calculate a timing average across the run. In any case, the simpler method used here should be sufficient to give us an indication of the comparative performance.

Camera segmentation

Here we are looking to find one or more humans in the frame and in the above video we can see a comparison of the example running on Jetson Orin Nano and AGX Orin. In practice, it seems to run just as well on either, although the frame rate was higher with AGX Orin.

If we compare the timing reports we see that it takes just short of 4ms total GPU processing time with Orin Nano, compared with a little over 3ms total processing time with AGX Orin. This is pretty fast and the GPU is clearly not being stressed with either. We can also see CPU processing times and as we’d expect, these are very similar and the GPU is doing all the heavy lifting here.

Background removal

While this example sounds quite simple the reality is that it takes quite a bit more processing than the previous example. Above we can see this being run on both dev kits and once again both platforms cope admirably. One thing we should note at this point is that the camera setup was far from ideal, with a lot of bright light coming in through the window behind the target object — the author! — which just as with photography, presents something of a challenge.

When comparing the timing reports this time we see that there is much more of a difference, with a total GPU processing time of around 52ms with Orin Nano vs. ~17ms with AGX Orin — which represents a threefold performance increase!

If we compare the frame rate achieved with both dev kits this also confirms a major performance increase, with around 30 FPS for Orin Nano vs. 60 FPS for AGX Orin at the time of capturing the above screenshots. Note that these figures, as with timing reports, jump around somewhat as the scene changes, so we’re talking about indicative figures.

Camera detection

With the final example, we’re detecting and classifying objects. Once again we see excellent performance with both the entry-level and flagship Jetson Orin dev kits.

The timing reports show AGX Orin clearly has the higher performance, but as with the first example, the total GPU processing time is very low and the performance difference is not quite as significant as with the background removal example.

Conclusions

The target application is clearly a key factor and if you’re getting fantastic performance with Jetson Orin Nano, you’re possibly not going to see stellar performance gains by moving up to AGX Orin. Indeed it may even be that you’re close to shifting data in and out of the GPU as fast as possible, or that any bottlenecks are elsewhere, such as in a camera or data acquisition system.

We’ve not scratched the surface of what is possible in terms of AI and wider CUDA accelerated applications, but where these place more extreme demands upon the GPU it’s clear that they will benefit from the extra processing power provided by AGX Jetson. This could even be seen where we had an application such as background removal, which worked great with both platforms, but where the measured performance with AGX Orin was two to three times that of Orin Nano. It would appear reasonable to assume therefore that this performance gap will only increase with particularly demanding applications running on the 40 TOPS Orin Nano vs. the 275 TOPS AGX Orin.

Final words

In part 1 of this series, we compared the base SoM specifications for Orin Nano and AGX Orin, along with things such as the video encoding/decoding capabilities, camera and PCIe etc. connectivity. Also the form factor, SoM/mainboard interfacing and power requirements. Noting how there are more factors to consider, alongside GPU/AI performance, when making a decision between the two platforms. And as is generally the case, it’s a question of the right tool for the job.

  — Andrew Back