How to Run Object Detection Models on Jetson or Raspberry Pi Using the ailia SDK

You can run YOLOX, YOLOv5, and other object detection models on Jetson or Raspberry Pi by installing the platform-specific ailia SDK wheel, letting the auto-detection logic in util/arg_utils.py select GPU for Jetson or CPU for Raspberry Pi, and executing the model script with standard CLI arguments.

The axinc-ai/ailia-models repository provides production-ready object detection implementations optimized for edge devices. Whether you are deploying on an NVIDIA Jetson Nano or a Raspberry Pi 4, the ailia SDK automatically handles runtime selection and model provisioning, allowing you to run object detection models on Jetson or Raspberry Pi using the ailia SDK with minimal configuration.

Platform-Specific Environment Selection

The ailia SDK exposes multiple runtime environments including CPU, CUDA, OpenCL, and Vulkan. The repository automatically selects the optimal backend based on the underlying hardware through logic defined in util/arg_utils.py.

On Jetson devices (Nano, TX2, Xavier), the script detects the aarch64 architecture but does not find the "rasp" platform identifier. Consequently, default_env_id remains set to the value returned by ailia.get_gpu_environment_id(), enabling GPU acceleration via CUDA or OpenCL.

On Raspberry Pi (Raspbian / Raspberry Pi OS), the platform string contains "rasp" or "rpt-rpi". The detection logic forces default_env_id = ailia.ENVIRONMENT_AUTO, which routes execution to the CPU backend because Vulkan performance is typically slower on Pi hardware.

# From util/arg_utils.py

default_env_id = ailia.get_gpu_environment_id() if AILIA_EXIST else 0
if (platform.machine().startswith("arm") or platform.machine().startswith("aarch64")) \
   and ("rasp" in platform.platform().lower() or "rpt-rpi" in platform.platform().lower()):
    default_env_id = ailia.ENVIRONMENT_AUTO

You can override this auto-detection using the --env_id flag:

# Force CPU on Jetson

python yolox.py --env_id 0

# Force GPU on Jetson (usually env_id 1)

python yolox.py --env_id 1

Installing the ailia SDK on Jetson and Raspberry Pi

The ailia SDK provides platform-specific Python wheels for ARM architectures. Download the appropriate package from the ailia website and install it into your Python environment.

For Jetson (aarch64):

pip3 install ailia-1.2.9-cp36-cp36m-linux_aarch64.whl

For Raspberry Pi (armv7l):

pip3 install ailia-1.2.9-cp36-cp36m-linux_armv7l.whl

After installation, ensure the ailia library is importable and add the SDK lib directory to LD_LIBRARY_PATH if necessary. The repository scripts import ailia directly, so the SDK must be available in the Python path used to run the demos.

Preparing Model Files Automatically

Each detection script in the repository calls check_and_download_models from util/model_utils.py to handle model provisioning. This function checks for the required .opt.onnx and .opt.onnx.prototxt files in the current directory. If absent, it downloads them from the ailia cloud storage.

# From object_detection/yolox/yolox.py

REMOTE_PATH = 'https://storage.googleapis.com/ailia-models/yolox/'
check_and_download_models(WEIGHT_PATH, MODEL_PATH, REMOTE_PATH)

When you run a script for the first time, the console displays download progress for approximately 30 MB of model data. No manual download or conversion is required.

Running Object Detection Demos

The repository provides unified scripts for various models such as YOLOX, YOLOv5, and YOLOv7. The following examples use YOLOX as a representative implementation located at object_detection/yolox/yolox.py.

Jetson (GPU-accelerated):

python object_detection/yolox/yolox.py \
    -i input.jpg \
    -m yolox_s \
    -w txt

The script automatically selects the GPU environment (env_id 1) on Jetson devices.

Raspberry Pi (CPU):

python object_detection/yolox/yolox.py \
    -i input.jpg \
    -m yolox_nano \
    --env_id 0

For live video inference, replace the input file flag with -v and specify the device index. The util/webcamera_utils.py helper automatically selects the appropriate backend for USB or CSI cameras.

python object_detection/yolox/yolox.py -v 0 -m yolox_s

Customizing Thresholds and Input Resolution

You can tune inference parameters via command-line arguments to balance speed and accuracy on resource-constrained devices.

• --threshold (-th): Confidence score cutoff (default 0.4).
• --iou: Non-maximum suppression (NMS) IoU threshold (default 0.45).
• --detection_width and --detection_height: Override the model's default input size.

For example, to reduce memory usage and increase FPS on a Jetson Nano:

python object_detection/yolox/yolox.py \
    -i video.mp4 \
    -m yolox_tiny \
    --detection_width 320 --detection_height 320 \
    -th 0.3 -iou 0.4

Core Inference Architecture

All detection scripts in the repository follow a consistent execution pipeline defined in the source code:

1. Argument parsing (util/arg_utils.py): Parses CLI options and determines the runtime environment.
2. Model provisioning (util/model_utils.py): Downloads ONNX and prototxt files if missing.
3. Environment initialization: Creates an ailia.Detector (high-level API) or ailia.Net (raw network) using the selected env_id.
4. Pre-processing: Resizes input images using model-specific utilities (e.g., yolox_utils).
5. Inference: Executes detector.run or detector.compute depending on the API level.
6. Post-processing: Applies NMS and converts outputs to bounding-box objects.
7. Visualization: Renders results and saves to disk or video.

This architecture is platform-agnostic; the only hardware-specific branch occurs during the initial environment selection in util/arg_utils.py.

Complete Example Commands

Jetson: GPU Inference on Video with JSON Output

python object_detection/yolox/yolox.py \
    -v sample.mp4 \
    -m yolox_s \
    --env_id 1 \
    -th 0.5 -iou 0.5 \
    -w json \
    -s output_video.mp4

Raspberry Pi: CPU Inference on Single Image

python object_detection/yolox/yolox.py \
    -i selfie.jpg \
    -m yolox_nano \
    --env_id 0 \
    -th 0.3 -iou 0.4 \
    -s result.jpg

Both commands execute the following actions automatically:

• Download yolox_nano.opt.onnx and its prototxt from https://storage.googleapis.com/ailia-models/yolox/ if not present locally.
• Initialize the runtime using --env_id (GPU on Jetson, CPU on Pi).
• Process the input through detector.run or detector.compute.
• Save visual results to the specified output path and write optional prediction files if -w is provided.

Troubleshooting and Performance Tips

Issue	Cause	Solution
"Vulkan not supported" error on Jetson	Jetson drivers may not expose Vulkan; ailia falls back to CUDA/OpenCL.	Omit --env_id to allow automatic selection of the best GPU backend, or ensure CUDA libraries are in LD_LIBRARY_PATH.
Low FPS on Raspberry Pi	CPU inference at full resolution (640×640) is computationally heavy.	Use the nano or tiny model variants (-m yolox_nano) and reduce input size with --detection_width 320.
Model download stalls	Network connectivity issues or SSL verification failures.	The downloader falls back to HTTP if SSL fails. Verify internet access or manually download from the Google Cloud Storage URL printed in the error message.
Camera not opened	Incorrect device index for CSI or USB cameras.	On Jetson, use -v 0 for CSI cameras; on Pi, ensure v4l2 drivers are installed for USB webcams.
Missing Detector API	ailia SDK version older than 1.2.9.	Upgrade to ailia SDK ≥ 1.2.9, or run without the -dt flag to use the raw ailia.Net interface.

Extending to Other Detection Models

All detection demos in the repository—including yolov5.py, yolov7.py, yolov8.py, and yolov6.py—share the same execution skeleton:

1. Import ailia and utility modules from util/.
2. Call check_and_download_models with the model-specific REMOTE_PATH.
3. Instantiate either ailia.Detector (high-level API) or ailia.Net (raw network).
4. Execute the identical pre-processing, inference, and post-processing pipeline.

Therefore, the environment selection logic and installation procedures described in this guide apply unchanged to any model in the object_detection/ directory. Simply replace yolox with your desired model name in the command line arguments.

Summary

• Automatic platform detection in util/arg_utils.py configures GPU acceleration for Jetson and CPU inference for Raspberry Pi.
• Install the correct wheel for your architecture (linux_aarch64 for Jetson, linux_armv7l for Raspberry Pi).
• Model files download automatically via check_and_download_models from Google Cloud Storage on first run.
• Use --env_id to override the runtime backend if needed (0 for CPU, 1 for GPU).
• Optimize performance by selecting lightweight model variants (nano, tiny) and reducing input resolution with --detection_width.

Frequently Asked Questions

How does the ailia SDK detect whether to use GPU or CPU on edge devices?

The SDK relies on platform inspection logic in util/arg_utils.py. It checks platform.machine() for arm or aarch64 and platform.platform() for the substring "rasp" or "rpt-rpi". Raspberry Pi matches force ailia.ENVIRONMENT_AUTO (CPU), while Jetson devices proceed to use ailia.get_gpu_environment_id() for CUDA or OpenCL acceleration.

Can I run object detection on a live camera feed using the ailia SDK?

Yes. Replace the -i (input file) flag with -v (video device) followed by the camera index. For example, python object_detection/yolox/yolox.py -v 0 -m yolox_s captures from the default camera. The util/webcamera_utils.py module automatically handles USB webcams on Raspberry Pi and CSI cameras on Jetson.

What should I do if inference is too slow on my Raspberry Pi 4?

Reduce computational load by using a smaller model variant and lower input resolution. Specify -m yolox_nano or -m yolox_tiny instead of the standard yolox_s, and add --detection_width 320 --detection_height 320 to process smaller images. This typically increases FPS from single digits to usable real-time rates on Raspberry Pi hardware.

Is it necessary to manually download ONNX model files before running the scripts?

No. The scripts automatically handle model provisioning through the check_and_download_models function in util/model_utils.py. On first execution, the script downloads the required .opt.onnx and .opt.onnx.prototxt files from https://storage.googleapis.com/ailia-models/<model>/ if they are not present in the local directory. Ensure your device has internet connectivity for the initial download.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}