# How to Integrate Custom AI Inference Models into ApraPipes Transforms: A Complete Guide

> Integrate custom AI inference models into ApraPipes transforms. Learn to configure, load, and run your AI models within ApraPipes for powerful data processing. Get the complete guide.

- Repository: [Apra Labs/aprapipes](https://github.com/apra-labs/aprapipes)
- Tags: how-to-guide
- Published: 2026-02-25

---

**To integrate a custom AI inference model into ApraPipes, create a Props class for configuration, implement a Transform module that loads the model in `init()` and runs inference in `process()`, then register it in [`ModuleRegistrations.cpp`](https://github.com/apra-labs/aprapipes/blob/main/ModuleRegistrations.cpp) using `filePathProp` for the model path.**

ApraPipes is a high-performance C++ framework for building media processing pipelines. Integrating custom AI inference models into ApraPipes transforms allows you to embed ONNX Runtime, TensorRT, or PyTorch models directly into your declarative pipelines. This guide walks through the exact implementation pattern used by built-in modules like `AudioToTextXForm` and `FaceDetectorXform` in the `apra-labs/aprapipes` repository.

## Understanding the ApraPipes Module Architecture

ApraPipes treats every processing step as a **Module**. A **Transform** module consumes frames, runs user-defined logic such as AI inference, and emits new frames. The architecture separates configuration from implementation through **Props** classes that describe configurable parameters like `modelPath` and `batchSize`.

Built-in transforms demonstrate this pattern clearly. `AudioToTextXForm` loads OpenAI's Whisper model in `init()`, runs speech recognition in `process()`, and emits text frames. `FaceDetectorXform` uses OpenCV DNN to load Caffe models for face detection. Your custom implementation will follow identical lifecycle methods.

## Step-by-Step Implementation Guide

### Step 1 – Define the Props Class

Create a Props class that inherits from `ModuleProps` to expose configuration options. Use the property-binding helpers so the declarative pipeline can populate them from JSON.

In [`base/include/MyModelXForm.h`](https://github.com/apra-labs/aprapipes/blob/main/base/include/MyModelXForm.h), define properties with `applyProperties` and `getProperty` methods:

```cpp
class MyModelXFormProps : public ModuleProps {
public:
    std::string modelPath;
    int batchSize = 1;

    template <typename PropsT>
    static void applyProperties(PropsT& props,
        const std::map<std::string, apra::ScalarPropertyValue>& values,
        std::vector<std::string>& missingRequired) {
        apra::applyProp(props.modelPath, "modelPath", values, true, missingRequired);
        apra::applyProp(props.batchSize, "batchSize", values, false, missingRequired);
    }

    apra::ScalarPropertyValue getProperty(const std::string& name) const {
        if (name == "modelPath") return modelPath;
        if (name == "batchSize") return static_cast<int64_t>(batchSize);
        throw std::runtime_error("Unknown property: " + name);
    }
};

```

Reference [`AudioToTextXForm.h`](https://github.com/apra-labs/aprapipes/blob/main/AudioToTextXForm.h) in the repository to see the complete `AudioToTextXFormProps` definition and the `applyProperties` template pattern.

### Step 2 – Implement the Transform Class

Implement the Transform class in [`base/src/MyModelXForm.cpp`](https://github.com/apra-labs/aprapipes/blob/main/base/src/MyModelXForm.cpp) by deriving from `Module`. Store a `Detail` object that holds the model handle, and implement the lifecycle methods:

- **`init()`** – Load the model using the path from Props (e.g., ONNX Runtime, TensorRT).
- **`process()`** – Convert incoming frames to input tensors, run inference, decode results, and push output frames.
- **`validateInputPins()` / `validateOutputPins()`** – Declare expected frame types (e.g., `RawImage`, `Tensor`, `Text`).

```cpp
bool MyModelXForm::init() {
    Ort::SessionOptions opts;
    opts.SetIntraOpNumThreads(1);
    mDetail->session = new Ort::Session(mDetail->ortEnv,
        mDetail->props.modelPath.c_str(), opts);
    return Module::init();
}

bool MyModelXForm::process(frame_container& frames) {
    auto inFrame = frames.begin()->second;
    
    // Convert frame to tensor and run inference
    std::array<const char*, 1> inputNames = {"input"};
    std::array<const char*, 1> outputNames = {"output"};
    auto outputTensors = mDetail->session->Run(Ort::RunOptions{}, 
        inputNames.data(), &inputTensor, 1, outputNames.data(), 1);
    
    // Decode and emit result
    std::string result = decodeOutput(outputTensors.front());
    auto outFrame = makeFrame(result.size());
    memcpy(outFrame->data(), result.c_str(), result.size());
    frames.insert({mDetail->outputPinId, outFrame});
    send(frames);
    return true;
}

```

Study [`AudioToTextXForm.cpp`](https://github.com/apra-labs/aprapipes/blob/main/AudioToTextXForm.cpp) to see how `whisper_init_from_file_with_params` loads the Whisper model and how the `process()` method handles inference.

### Step 3 – Register the Module

Register the new module in [`base/src/declarative/ModuleRegistrations.cpp`](https://github.com/apra-labs/aprapipes/blob/main/base/src/declarative/ModuleRegistrations.cpp). Add a block similar to `AudioToTextXForm`, specifying category, description, input/output pins, and property definitions. Use `filePathProp` with `PathRequirement::MustExist` to ensure the model file is present at pipeline load time.

```cpp
if (!registry.hasModule("MyModelXForm")) {
    registerModule<MyModelXForm, MyModelXFormProps>()
        .category(ModuleCategory::Transform)
        .description("Runs custom AI inference (ONNX/TensorRT) on input frames")
        .tags("transform", "ai", "inference", "onnx", "ml")
        .input("input", "RawImage")
        .output("output", "Text")
        .filePathProp("modelPath", "Path to the model file (ONNX / TensorRT)",
                      PathRequirement::MustExist, true)
        .intProp("batchSize", "Batch size for inference", false, 1, 1, 128)
        .selfManagedOutputPins();
}

```

Lines 1042-1054 in [`ModuleRegistrations.cpp`](https://github.com/apra-labs/aprapipes/blob/main/ModuleRegistrations.cpp) demonstrate the registration pattern for `AudioToTextXForm` with `filePathProp`.

### Step 4 – Add Documentation and Tests

Create documentation in [`docs/declarative-pipeline/MyModelXForm.md`](https://github.com/apra-labs/aprapipes/blob/main/docs/declarative-pipeline/MyModelXForm.md) describing the required properties and example pipeline JSON. Update [`docs/declarative-pipeline/PROGRESS.md`](https://github.com/apra-labs/aprapipes/blob/main/docs/declarative-pipeline/PROGRESS.md) to list the new module so the web UI picks it up.

Add a unit test in `base/test/` mirroring existing XForm tests like [`audioToTextXform_tests.cpp`](https://github.com/apra-labs/aprapipes/blob/main/audioToTextXform_tests.cpp). The test should load a tiny test model and verify output frame generation. Re-run the build with `cmake && cmake --build build` to verify compilation across all CI workflows (Linux, Windows, macOS, ARM64).

## How the Components Work Together

When a JSON pipeline is parsed, the **ModuleFactory** looks up the module name (`MyModelXForm`) in the **ModuleRegistry** populated by [`ModuleRegistrations.cpp`](https://github.com/apra-labs/aprapipes/blob/main/ModuleRegistrations.cpp). The JSON object's `modelPath` field maps to `MyModelXFormProps::modelPath` via the `applyProperties` template, ensuring type-safe configuration binding.

At runtime, `init()` executes once per module instance, loading the model from the supplied path—mirroring how `AudioToTextXForm` loads Whisper via `whisper_init_from_file_with_params`. For each incoming frame, `process()` converts the frame to the model's input tensor, executes inference, decodes the results, and forwards the output frame downstream using `send(frames)`.

## Key Files to Reference

- **[`base/include/AudioToTextXForm.h`](https://github.com/apra-labs/aprapipes/blob/main/base/include/AudioToTextXForm.h)** – Props definition and `applyProperties` template pattern
- **[`base/src/AudioToTextXForm.cpp`](https://github.com/apra-labs/aprapipes/blob/main/base/src/AudioToTextXForm.cpp)** – Model loading (`whisper_init_from_file_with_params`) and inference flow in `process()`
- **[`base/src/declarative/ModuleRegistrations.cpp`](https://github.com/apra-labs/aprapipes/blob/main/base/src/declarative/ModuleRegistrations.cpp)** – Module registration with `filePathProp` and `PathRequirement::MustExist` (lines 1042-1054)
- **[`base/src/FaceDetectorXform.cpp`](https://github.com/apra-labs/aprapipes/blob/main/base/src/FaceDetectorXform.cpp)** – Deep learning model loading (Caffe) using OpenCV DNN
- **[`docs/declarative-pipeline/PROGRESS.md`](https://github.com/apra-labs/aprapipes/blob/main/docs/declarative-pipeline/PROGRESS.md)** – Master list of registered modules for UI integration

## Summary

- **Create a Props class** inheriting from `ModuleProps` with `modelPath` and configuration fields, implementing `applyProperties` for JSON binding.
- **Implement the Transform** by deriving from `Module`, loading the model in `init()`, and running inference in `process()` using your chosen SDK (ONNX Runtime, TensorRT, etc.).
- **Register the module** in [`ModuleRegistrations.cpp`](https://github.com/apra-labs/aprapipes/blob/main/ModuleRegistrations.cpp) using `filePathProp` with `PathRequirement::MustExist` to ensure the model file is available at pipeline load time.
- **Document and test** by adding markdown documentation in `docs/declarative-pipeline/` and unit tests in `base/test/` to verify functionality across all supported platforms.

## Frequently Asked Questions

### What inference engines are supported for custom AI models in ApraPipes?

ApraPipes supports any inference engine that provides a C++ API. The framework itself is agnostic to the backend implementation. You can integrate **ONNX Runtime**, **TensorRT**, **OpenCV DNN**, **PyTorch C++ (LibTorch)**, or proprietary SDKs by including their headers in your Transform implementation and linking the appropriate libraries in your CMake configuration.

### How do I handle different input frame types in my custom transform?

Override the `validateInputPins()` method to declare acceptable frame types. For example, return `meta->getFrameType() == FrameMetadata::RAW_IMAGE` for image inputs or `FrameMetadata::AUDIO` for audio. In `process()`, cast the incoming frame to the appropriate type using `getFrame()` or similar accessors. If your model requires tensor inputs, convert the frame data using utility functions from `FrameMetadata` or [`Utils.h`](https://github.com/apra-labs/aprapipes/blob/main/Utils.h) before feeding it to the inference session.

### Can I use GPU acceleration for custom AI inference in ApraPipes?

Yes. When registering your module in [`ModuleRegistrations.cpp`](https://github.com/apra-labs/aprapipes/blob/main/ModuleRegistrations.cpp), use the `registerCudaModule` helper instead of `registerModule` if your implementation requires CUDA streams. In your `Detail` class or `init()` method, configure your inference engine for GPU execution—for example, create an ONNX Runtime session with `OrtCUDAProviderOptions`, or initialize TensorRT with CUDA context. Ensure your `process()` method handles GPU memory appropriately and synchronizes CUDA streams before sending frames downstream.

### How do I debug inference errors in my custom transform?

Add detailed logging in the `init()` method to verify model loading success and path resolution. In `process()`, check the return values of inference API calls (e.g., `Ort::Session::Run`) and throw exceptions or return `false` to signal pipeline failure. Use the `validateInputPins()` and `validateOutputPins()` methods to catch metadata mismatches early. For runtime debugging, build with `cmake -DCMAKE_BUILD_TYPE=Debug` and use GDB or Visual Studio to step through the `process()` method, inspecting tensor shapes and frame data before and after inference.