How to Integrate ailia-models with Python, C++, Unity, and Rust: A Complete SDK Guide

To integrate ailia-models, install the language-specific ailia SDK, load the model's weights.bin and deploy.json files from the repository, preprocess inputs to CHW float32 format normalized to [0,1], and call the predict method to execute inference on images, video, or audio.

The axinc-ai/ailia-models repository provides production-ready implementations of neural networks for computer vision, audio processing, and multimodal AI. To integrate ailia-models with Python, C++, Unity, and Rust, developers use the ailia SDK runtime, which exposes a unified cross-language API for model loading and tensor inference.

Prerequisites and SDK Overview

Before writing integration code, download the appropriate ailia SDK for your target language:

• Python: pip3 install ailia

• C++: Static/dynamic libraries from the official C++ SDK

• Unity: C# package via Unity Package Manager

• Rust: ailia-sdk crate on crates.io

Each model in the repository includes a weights.bin file (trained parameters) and a deploy.json file (network architecture). You will load these using the SDK's Net or Network class.

Python Integration

Python provides the fastest path to running ailia-models, with ready-to-execute scripts for every model in the repository.

Installation and Quick Start

Install the SDK and optional dependencies:

pip3 install ailia
pip install opencv-python numpy

Run a model using the provided script. For example, YOLOv3-tiny object detection:

python3 object_detection/yolov3-tiny/yolov3-tiny.py -i input.jpg -s output.jpg

Reference implementation: [object_detection/yolov3-tiny/yolov3-tiny.py](https://github.com/axinc-ai/ailia-models/blob/master/object_detection/yolov3-tiny/yolov3-tiny.py)

Programmatic API Usage

For custom applications, use the ailia Python API directly:

import ailia
import numpy as np

# 1️⃣ Load the model

net = ailia.Net(
    "object_detection/yolov3-tiny/weights/yolov3-tiny.onnx.prototxt",
    "object_detection/yolov3-tiny/weights/yolov3-tiny.onnx"
)

# 2️⃣ Preprocess input: HWC → CHW, float32, [0,1]

image = ailia.imread("input.jpg")
data = ailia.resize(image, (416, 416))
data = ailia.transpose(data, (2, 0, 1))  # CHW

data = data.astype(np.float32) / 255.0
data = np.expand_dims(data, axis=0)      # Add batch dimension

# 3️⃣ Inference

net.predict(data)

# 4️⃣ Post-process outputs

boxes = net.get_blob_data(net.find_blob_index_by_name("boxes"))
scores = net.get_blob_data(net.find_blob_index_by_name("scores"))

Key helper files: [util/webcamera_utils.py](https://github.com/axinc-ai/ailia-models/blob/master/util/webcamera_utils.py) for live camera input handling.

C++ Integration

The C++ SDK provides static and dynamic libraries for high-performance inference. A companion repository, ailia-models-cpp, contains wrapper implementations for many models.

SDK Setup

Download the C++ SDK from the official documentation: https://ailia-ai.github.io/ailia-sdk/api/cpp/en/

Clone the C++ examples repository:

git clone https://github.com/ailia-ai/ailia-models-cpp.git

Copy the model files (weights.bin and deploy.json) from the main ailia-models repository into your C++ project directory.

Implementation Example

#include <ailia.h>
#include <opencv2/opencv.hpp>
#include <iostream>

int main()
{
    // 1️⃣ Initialise environment (GPU 0)
    AILIAEnvironment* env = nullptr;
    ailiaCreate(&env, AILIA_ENVIRONMENT_TYPE_GPU, 0);

    // 2️⃣ Create network instance
    AILIANetwork* net = nullptr;
    ailiaCreateNetwork(&net, env);

    // 3️⃣ Load model weights and architecture
    ailiaLoadWeightFileA(net, "yolov3-tiny/weights/weights.bin");
    ailiaLoadModelFileA(net, "yolov3-tiny/weights/deploy.json");

    // 4️⃣ Load and preprocess image (OpenCV)
    cv::Mat img = cv::imread("input.jpg");
    cv::Mat resized;
    cv::resize(img, resized, cv::Size(416, 416));
    
    // Convert to CHW float32 [0,1]
    cv::Mat floatImg;
    resized.convertTo(floatImg, CV_32FC3, 1.0/255.0);
    std::vector<float> inputData;
    inputData.assign((float*)floatImg.data, (float*)floatImg.data + 416*416*3);
    
    // Transpose HWC to CHW manually or use SDK helper
    // ...

    // 5️⃣ Inference
    AILIAShape inputShape = {1, 3, 416, 416};
    ailiaPredict(net, inputData.data(), inputShape);

    // 6️⃣ Get outputs
    AILIAShape outputShape;
    ailiaGetOutputShape(net, &outputShape, 0);
    std::vector<float> outputData(outputShape.x * outputShape.y * outputShape.z * outputShape.w);
    ailiaGetOutputData(net, 0, outputData.data());

    return 0;
}

Note: The C++ SDK uses raw ailia.h C API or the C++ wrapper classes (ailia::Network). Refer to the ailia-models-cpp repository for complete build scripts and CMake configurations.

Unity (C#) Integration

Unity developers use the ailia C# package to run models on desktop, mobile, and embedded platforms.

Installation

Add the package via Unity Package Manager:

1. Open Window → Package Manager
2. Click + → Add package from git URL…
3. Enter: https://github.com/ailia-ai/ailia-models-unity.git

Alternatively, download the .unitypackage from the releases page.

C# Implementation

using UnityEngine;
using Ailia;  // ailia SDK namespace

public class AiliaDetector : MonoBehaviour
{
    [SerializeField] private TextAsset modelWeights;  // .bin file
    [SerializeField] private TextAsset modelProto;    // .json or .onnx.prototxt
    [SerializeField] private Texture2D inputImage;

    private Net neuralNet;
    private Environment environment;

    void Start()
    {
        // 1️⃣ Initialise environment (GPU 0, or -1 for CPU)
        environment = new Environment(0);
        neuralNet = new Net(environment);

        // 2️⃣ Load model from TextAsset bytes
        neuralNet.LoadModel(modelWeights.bytes, modelProto.text);

        // 3️⃣ Convert Texture2D to Tensor
        Tensor inputTensor = Tensor.FromTexture2D(inputImage);
        
        // 4️⃣ Preprocess: HWC → CHW, float32, normalize
        inputTensor = inputTensor.Transpose(new[] { 2, 0, 1 });
        inputTensor = inputTensor.CastToFloat();
        inputTensor = inputTensor.Div(255.0f);

        // 5️⃣ Inference
        neuralNet.Predict(inputTensor);

        // 6️⃣ Retrieve outputs by blob name
        Tensor boxes = neuralNet.GetOutput("boxes");
        Tensor scores = neuralNet.GetOutput("scores");
        Tensor classIds = neuralNet.GetOutput("classes");

        // Convert to arrays for Unity UI rendering
        float[] boxData = boxes.ToArray();
        Debug.Log($"Detected {boxData.Length / 4} objects");
    }

    void OnDestroy()
    {
        neuralNet?.Dispose();
        environment?.Dispose();
    }
}

Reference: The ailia-models-unity repository contains sample scenes demonstrating camera input and real-time processing. See the Unity-specific section in [TUTORIAL.md](https://github.com/axinc-ai/ailia-models/blob/master/TUTORIAL.md) for platform-specific build instructions.

Rust Integration

Rust developers can integrate ailia-models using the ailia-sdk crate, which provides safe bindings to the native runtime.

Setup

Add the crate to your Cargo.toml:

[dependencies]
ailia-sdk = "0.1"
opencv = "0.71"  # Optional, for image I/O

anyhow = "1.0"   # For error handling

Clone the Rust examples repository:

git clone https://github.com/ailia-ai/ailia-models-rust.git

Rust Implementation

use ailia_sdk::{Environment, Net, Tensor};
use opencv::{imgcodecs, imgproc, prelude::*};
use anyhow::Result;

fn main() -> Result<()> {
    // 1️⃣ Initialise environment (GPU 0)
    let env = Environment::new(0)?;
    
    // 2️⃣ Create network and load model files
    let mut net = Net::new(&env);
    net.load_model(
        "object_detection/yolov3-tiny/weights/yolov3-tiny.onnx",
        "object_detection/yolov3-tiny/weights/yolov3-tiny.onnx.prototxt"
    )?;
    
    // 3️⃣ Load and preprocess image (OpenCV)
    let img = imgcodecs::imread("input.jpg", imgcodecs::IMREAD_COLOR)?;
    let mut resized = Mat::default();
    imgproc::resize(
        &img, 
        &mut resized, 
        opencv::core::Size::new(416, 416), 
        0.0, 0.0, 
        imgproc::INTER_LINEAR
    )?;
    
    // 4️⃣ Convert to ailia Tensor: HWC → CHW, float32, normalize [0,1]
    let tensor = Tensor::from_mat(&resized)?
        .to_f32()?
        .div_scalar(255.0);
    
    // 5️⃣ Run inference
    net.predict(&tensor)?;
    
    // 6️⃣ Retrieve output tensors by name
    let boxes = net.get_output("boxes")?;
    let scores = net.get_output("scores")?;
    let classes = net.get_output("classes")?;
    
    // Process results...
    println!("Inference complete. Output shape: {:?}", boxes.shape());
    
    Ok(())
}

Reference: The ailia-models-rust repository contains complete Cargo projects demonstrating error handling and batch processing.

Input Preprocessing and Output Handling

All language bindings require identical tensor formatting to interface with ailia-models.

Standard Input Pipeline

Regardless of language, transform your raw data to match this specification:

1. Resize to model input dimensions (e.g., 416×416 for YOLOv3-tiny)
2. Transpose from HWC (Height-Width-Channels) to CHW format
3. Cast to float32 (FP32)
4. Normalize pixel values to [0, 1] by dividing by 255.0

Output Retrieval

After calling predict, access output tensors by the blob name defined in the model's deploy.json:

• Python: net.get_blob_data(net.find_blob_index_by_name("boxes"))
• C++: net.getOutput("boxes")
• Unity: neuralNet.GetOutput("boxes")
• Rust: net.get_output("boxes")

Summary

• Install the ailia SDK for your target language via pip, NuGet, Cargo, or direct download.
• Obtain model files (weights.bin and deploy.json) from the axinc-ai/ailia-models repository.
• Preprocess all inputs to CHW float32 format normalized to [0,1] before inference.
• Load models using Net (Python/C#/Rust) or Network (C++) classes, specifying GPU or CPU environment.
• Run inference via the predict method and retrieve output tensors by name for post-processing.

Frequently Asked Questions

How do I switch between GPU and CPU inference in ailia-models?

Set the environment ID when initializing the Environment class. Use 0 for the first GPU, or -1 to force CPU execution. In Python, use ailia.set_gpu_environment_id(0); in C++, pass the ID to ailia::Environment env(0); in Unity and Rust, use new Environment(0) and Environment::new(0) respectively.

Where do I find the model weight files for integration?

Model weights and architecture files reside in individual subdirectories within the axinc-ai/ailia-models repository. Each model folder contains a weights directory with .bin (weights) and .json or .onnx.prototxt (graph definition) files. For example, YOLOv3-tiny files are located at object_detection/yolov3-tiny/weights/.

What image format does the ailia SDK expect for computer vision models?

The SDK requires CHW (Channels-Height-Width) layout with float32 precision. Preprocess by resizing to the model's input size (e.g., 416×416), transposing axes from HWC to CHW, casting to float32, and dividing pixel values by 255.0 to normalize to the [0,1] range.

Can I use ailia-models in commercial applications?

Yes. The ailia SDK and ailia-models repository are designed for commercial use. The models are provided under various open-source licenses (depending on the specific model), while the ailia SDK runtime is proprietary but royalty-free for deployment. Check individual model README.md files for specific license terms.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}