How to Define and Use Custom FrameMetadata Types in ApraPipes: A Complete Guide

To define custom FrameMetadata types in ApraPipes, subclass FrameMetadata, implement virtual methods like reset() and isSet(), add a nested Metadata struct with registration constants, and use the REGISTER_FRAME_TYPE macro to enroll the type in the global registry.

ApraPipes treats every data packet flowing through a pipeline as a frame. Each frame carries a pointer to a FrameMetadata object that describes the data type, memory layout, and size. While the core library provides built-in metadata classes like RawImageMetadata and H264Metadata in base/include/RawImageMetadata.h and base/include/H264Metadata.h, you will often need to attach application-specific context—such as sensor IDs, GPS coordinates, or detection scores—that these built-in types cannot hold. Creating custom FrameMetadata types allows you to enrich frame data while maintaining full compatibility with the declarative pipeline system and runtime type safety.

Understanding the FrameMetadata Architecture

The Base FrameMetadata Class

All metadata in ApraPipes inherits from apra::FrameMetadata, defined in base/include/FrameMetadata.h. This abstract base class establishes the interface that every metadata type must implement. The core responsibilities include tracking whether the metadata has been initialized (isSet()), resetting fields to a default state (reset()), and optionally reporting the size of associated binary data (getDataSize()).

When you create a custom type, you must override these virtual methods to ensure the pipeline can correctly manage memory and validate frame state.

Built-in Metadata Types as Reference

The ApraPipes source code provides excellent reference implementations in base/include/RawImageMetadata.h and base/include/H264Metadata.h. These classes demonstrate how to:

• Inherit from FrameMetadata and call the base constructor with an appropriate FrameType.
• Add domain-specific fields (e.g., width, height, direction, mp4Seek).
• Implement reset() to clear custom fields back to sentinel values like NOT_SET_NUM.
• Provide convenience getters and setters for type-safe access.

Studying these files before writing your own implementation ensures you follow the established patterns for memory management and field initialization.

Step-by-Step Guide to Creating Custom FrameMetadata Types

Step 1: Subclass FrameMetadata and Implement Virtual Methods

Create a new header file for your metadata class and inherit from FrameMetadata. You must implement reset() and isSet() to satisfy the interface. If your metadata will carry binary data, also implement getDataSize().

// SensorMetadata.h
#pragma once
#include "FrameMetadata.h"

class SensorMetadata : public FrameMetadata {
public:
    // Default constructor for registration
    SensorMetadata() : FrameMetadata(FrameType::GENERAL) {}
    
    // Parameterized constructor for actual use
    SensorMetadata(int w, int h, double ts, int id)
        : FrameMetadata(FrameType::GENERAL), 
          width(w), height(h), timestamp(ts), sensorId(id) {}

    // Required: Reset all fields to default state
    void reset() override {
        FrameMetadata::reset();
        width = NOT_SET_NUM;
        height = NOT_SET_NUM;
        timestamp = 0.0;
        sensorId = -1;
    }

    // Required: Check if metadata has been initialized
    bool isSet() override { 
        return width != NOT_SET_NUM; 
    }

    // Optional: Report size of binary data if applicable
    size_t getDataSize() override { 
        return sizeof(width) + sizeof(height) + 
               sizeof(timestamp) + sizeof(sensorId); 
    }

    // Custom getters
    int getWidth() const { return width; }
    int getHeight() const { return height; }
    double getTimestamp() const { return timestamp; }
    int getSensorId() const { return sensorId; }

private:
    int width = NOT_SET_NUM;
    int height = NOT_SET_NUM;
    double timestamp = 0.0;
    int sensorId = -1;
};

Step 2: Add the Nested Metadata Struct for Registration

To enable the declarative pipeline system to recognize your type, add a public nested struct named Metadata with static constexpr members. This struct supplies compile-time constants used by the REGISTER_FRAME_TYPE macro defined in base/include/declarative/FrameTypeRegistry.h.

Add this inside your class definition:

struct Metadata {
    static constexpr std::string_view name = "SensorMetadata";
    static constexpr std::string_view parent = "FRAME";  // Inherits from generic FRAME
    static constexpr std::string_view description = "Metadata for sensor-derived frames with GPS and timestamp";
    static constexpr std::array<std::string_view, 2> tags = { "sensor", "enriched" };
};

The parent field establishes inheritance in the type hierarchy. Use "FRAME" for top-level custom types, or reference another registered type like "RawImageMetadata" if your metadata extends a specific built-in class.

Step 3: Register the Type with REGISTER_FRAME_TYPE

Place the registration macro in a source file (.cpp) that is always compiled with your application. This creates a static initialization object that adds your type to the global apra::FrameTypeRegistry at program startup.

// SensorMetadata.cpp
#include "SensorMetadata.h"

REGISTER_FRAME_TYPE(SensorMetadata);

Do not place this macro in a header file, as it would create multiple registration entries if the header is included in multiple translation units.

Using Custom FrameMetadata in Pipeline Modules

Attaching Metadata to Output Pins

When a module declares an output pin, it passes a metadata instance that describes the frames that pin will produce. Downstream modules use this metadata to validate compatibility and allocate resources.

class SensorProducer : public Module {
public:
    SensorProducer() {
        // Create metadata instance
        auto meta = framemetadata_sp(
            new SensorMetadata(1920, 1080, 0.0, 42)
        );
        
        // Register output pin with this metadata
        addOutputPin(meta);
    }
    
    bool produce(frame_sp& out) override {
        // Update timestamp dynamically
        auto meta = FrameMetadataFactory::downcast<SensorMetadata>(
            getOutputMetadata(0)
        );
        meta->timestamp = std::chrono::duration<double>(
            std::chrono::steady_clock::now().time_since_epoch()
        ).count();
        
        // Create frame and attach metadata
        out = makeFrame(meta->getDataSize(), "output0");
        out->setMetadata(meta);
        return true;
    }
};

Safely Retrieving and Downcasting Metadata

Downstream modules receive generic frame_sp objects. To access your custom fields, retrieve the metadata pointer and downcast it using the type-safe helper in base/include/FrameMetadataFactory.h.

class SensorConsumer : public Module {
public:
    SensorConsumer() {
        // Declare expected input type
        addInputPin(framemetadata_sp(new SensorMetadata()));
    }
    
    bool process(frame_container& input, frame_sp& out) override {
        // Retrieve frame by type name
        auto frame = Module::getFrameByType(
            input, 
            SensorMetadata::Metadata::name
        );
        
        // Safe downcast to custom type
        auto meta = FrameMetadataFactory::downcast<SensorMetadata>(
            frame->getMetadata()
        );
        
        // Access enriched data
        std::cout << "Sensor " << meta->getSensorId() 
                  << " @ " << meta->getTimestamp() << "s\n";
        
        // Pass through
        out = frame;
        return true;
    }
};

The FrameMetadataFactory::downcast<T>() function provides runtime type checking, ensuring you cannot accidentally interpret metadata as the wrong type.

Complete Working Example: SensorMetadata Implementation

Below is a minimal, self-contained implementation demonstrating the full lifecycle of a custom metadata type—from class definition through registration to production and consumption in pipeline modules.

// ------------------- SensorMetadata.h -------------------
#pragma once
#include "FrameMetadata.h"
#include "FrameMetadataFactory.h"

class SensorMetadata : public FrameMetadata {
public:
    // Constructors
    SensorMetadata() : FrameMetadata(FrameType::GENERAL) {}
    SensorMetadata(int w, int h, double ts, int id)
        : FrameMetadata(FrameType::GENERAL), width(w), height(h),
          timestamp(ts), sensorId(id) {}

    // Required overrides
    void reset() override {
        FrameMetadata::reset();
        width = NOT_SET_NUM;
        height = NOT_SET_NUM;
        timestamp = 0.0;
        sensorId = -1;
    }
    
    bool isSet() override { return width != NOT_SET_NUM; }
    
    size_t getDataSize() override { 
        return sizeof(width) + sizeof(height) + 
               sizeof(timestamp) + sizeof(sensorId); 
    }

    // Accessors
    int getWidth() const { return width; }
    int getHeight() const { return height; }
    double getTimestamp() const { return timestamp; }
    int getSensorId() const { return sensorId; }

    // Registration metadata
    struct Metadata {
        static constexpr std::string_view name = "SensorMetadata";
        static constexpr std::string_view parent = "FRAME";
        static constexpr std::string_view description = "Metadata for sensor-derived frames";
        static constexpr std::array<std::string_view, 2> tags = { "sensor", "custom" };
    };

private:
    int width = NOT_SET_NUM;
    int height = NOT_SET_NUM;
    double timestamp = 0.0;
    int sensorId = -1;
};

// ------------------- SensorMetadata.cpp -------------------
#include "SensorMetadata.h"

REGISTER_FRAME_TYPE(SensorMetadata);

// ------------------- ProducerModule.cpp -------------------
#include "SensorMetadata.h"
#include "Module.h"

class SensorProducer : public Module {
public:
    SensorProducer() {
        auto meta = framemetadata_sp(
            new SensorMetadata(1920, 1080, 0.0, 42)
        );
        addOutputPin(meta);
    }

    bool produce(frame_sp& out) override {
        auto meta = FrameMetadataFactory::downcast<SensorMetadata>(getOutputMetadata(0));
        meta->timestamp = std::chrono::duration<double>(
            std::chrono::steady_clock::now().time_since_epoch()
        ).count();

        out = makeFrame(meta->getDataSize(), "output0");
        out->setMetadata(meta);
        return true;
    }
};

// ------------------- ConsumerModule.cpp -------------------
#include "SensorMetadata.h"
#include "Module.h"
#include "FrameMetadataFactory.h"

class SensorConsumer : public Module {
public:
    SensorConsumer() {
        addInputPin(framemetadata_sp(new SensorMetadata()));
    }

    bool process(frame_container& input, frame_sp& out) override {
        auto frame = Module::getFrameByType(input, SensorMetadata::Metadata::name);
        auto meta = FrameMetadataFactory::downcast<SensorMetadata>(frame->getMetadata());

        std::cout << "Received frame from sensor " << meta->getSensorId()
                  << " at " << meta->getTimestamp() << "s\n";

        out = frame;
        return true;
    }
};

Summary

Creating custom FrameMetadata types in ApraPipes requires three essential steps:

• Inherit from FrameMetadata and implement the pure virtual methods reset() and isSet() to manage initialization state, plus getDataSize() if your metadata carries binary payload information.
• Embed a nested Metadata struct containing static constexpr fields (name, parent, description, tags) that the REGISTER_FRAME_TYPE macro consumes to populate the global FrameTypeRegistry.
• Use FrameMetadataFactory::downcast<T>() when retrieving metadata in downstream modules to safely convert the base pointer back to your specific type without risking undefined behavior.

Once registered via REGISTER_FRAME_TYPE, your custom metadata integrates seamlessly with the declarative JSON pipeline system, automatic documentation generation, and the existing module ecosystem.

Frequently Asked Questions

How do I ensure my custom FrameMetadata type is thread-safe?

The FrameMetadata base class itself does not enforce thread-safety; it stores only lightweight descriptors. When you add custom fields (e.g., timestamps, sensor IDs), treat metadata instances as immutable after they are attached to a frame, or protect mutable state with your own synchronization primitives if modules will modify metadata concurrently. The recommended pattern is to create a new metadata instance per frame rather than mutating shared metadata objects.

Can I inherit from existing metadata types like RawImageMetadata instead of FrameMetadata?

Yes. If your enriched data extends a specific built-in type (for example, adding GPS coordinates to image metadata), inherit from RawImageMetadata instead of the base FrameMetadata class. Update the nested Metadata struct’s parent field to "RawImageMetadata" so the registry reflects the inheritance hierarchy. This allows downstream modules to treat your type as both your specific class and as a raw image metadata where appropriate.

Where should I place the REGISTER_FRAME_TYPE macro to avoid linker errors?

Place the REGISTER_FRAME_TYPE(MyClass) macro in exactly one source file (.cpp), not in a header. The macro expands to a static object definition that executes registration code during program initialization. If placed in a header included by multiple translation units, you will encounter multiple definition linker errors. A dedicated MyCustomMetadata.cpp file that includes the header and contains only the registration line is the standard pattern.

How do I access custom metadata in Python bindings if I extend the framework?

The ApraPipes Python bindings expose metadata through the same FrameMetadata base interface. To access custom fields from Python, you must expose your specific metadata class using pybind11 (or the binding generator used by the project), providing property getters for your custom fields. Alternatively, serialize your enriched data into the frame's raw buffer and parse it on the Python side if you cannot extend the bindings. The C++ FrameMetadataFactory::downcast approach remains the most efficient method for C++ modules.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}