Real Time Visual Recognition

Note: This tutorial is applicable for Kerloud UAV products equipped with Jetson Nano only.

The official sample project to start with visual recognition is the jetson-inference repository in https://github.com/dusty-nv/jetson-inference, and it is a hello-world demo for Nvidia Jetson Nano development. The repo supports image classification, object localization and semantic segmentation. Users can delve into project details for their interests. Here we will illustrate the basic part of visual recognition only and address common problems while deploying these code.

Code Structure

Main directories are listed below, and readers can delve into the toplevel CMakeLists.txt for detailed understanding.

  • c/: network classes for detectNet, imageNet and segNet

  • data/: network data and images

  • docker/: scripts to pull, build and run docker images in containers.

  • doc/: documentations

  • examples/: main executables to deploy various networks.

  • plugin/, python/, tools/, utils/: other functions, libraries and scripts

How to Build

Users can refer to the official instructions for building this project in https://github.com/dusty-nv/jetson-inference/blob/master/docs/building-repo-2.md. However, due to the network problem in China, users might face interrupted behaviors when the code try to access blocked websites in github or nvidia. Hence we modified the code and downloaded it under the directory ~/jetson-inference with all network data as part of the repo. Users could then follow the commands below for easy build:

sudo apt-get update
sudo apt-get install git cmake libpython3-dev python3-numpy
cd ~/jetson-inference
git checkout master_ck
mkdir build
cd build
cmake ../
make -j2
sudo make install
sudo ldconfig

How to Run

The links on how to deploy the code are:

https://github.com/dusty-nv/jetson-inference/blob/master/docs/imagenet-console.md

https://github.com/dusty-nv/jetson-inference/blob/master/docs/imagenet-camera-2.md

To recognize a picture in folder, follow the commands:

cd ~/jetson-inference/build/aarch64/bin

./imagenet-console.py --network=googlenet images/orange_0.jpg output_0.jpg  # --network flag is optional (default is googlenet)

The commands will generate a new picture under the bin folder, labeled with the result and a confidence number, shown below for two objects:

../_images/output_0.jpg ../_images/output_1.jpg

For live video recognition: (Note that the onboard camera is installed on CSI port 0)

./imagenet csi://0                    # MIPI CSI camera
../_images/videocam_res.png

For better recognition results, clean background is highly suggested. The deployed googlenet doesn’t perform very well in video recognition.

References

https://developer.nvidia.com/embedded/twodaystoademo

https://developer.nvidia.com/embedded/learn/tutorials

https://developer.nvidia.com/embedded/learn/getting-started-jetson

https://developer.nvidia.com/embedded/community/support-resources