How the ModuleFactory and Registration Pattern Enable Dynamic Module Loading in ApraPipes

The ModuleFactory and registration pattern in ApraPipes enables dynamic module loading by using static initialization macros to register module metadata and factory lambdas into a central registry, allowing runtime instantiation from declarative JSON or TOML descriptions without recompiling the pipeline builder.

ApraPipes is a high-performance C++ framework for building video and image processing pipelines from declarative configuration files. At its core, the ModuleFactory and registration pattern provides the runtime flexibility to instantiate concrete module objects from string names defined in JSON or TOML, eliminating the need for hardcoded switch statements or manual factory methods when adding new processing nodes.

The Registration Mechanism: From Compile-Time to Run-Time

Every concrete module in ApraPipes registers itself using the REGISTER_MODULE(ModuleClass, PropsClass) macro defined in base/include/declarative/ModuleRegistry.h【/cache/repos/github.com/apra-labs/aprapipes/main/base/include/declarative/ModuleRegistry.h#L382-L464】. This macro executes during static initialization, ensuring that every linked module automatically populates the central registry before main() executes.

The macro performs four critical operations:

1. Metadata Collection – Copies the module's Metadata struct (containing name, category, pins, properties, and version) into a ModuleInfo object.
2. Factory Lambda Creation – Constructs a lambda that instantiates the concrete module from a std::map<std::string, ScalarPropertyValue>. The lambda uses SFINAE-aware tryApplyProperties to inject properties into the module's PropsClass, falling back to default construction if no properties are provided.
3. Registry Insertion – Calls ModuleRegistry::instance().registerModule(std::move(info)) to store the metadata and factory.
4. Re-registration Callback – Adds a callback via addRegistrationCallback(registerIfNeeded) to support test isolation, allowing test suites to clear the registry and repopulate it between test runs.

Because registration occurs at static initialization time, adding a new module requires only compiling and linking the translation unit. The pipeline construction code remains unchanged.

ModuleFactory: Runtime Builder from Registry Metadata

While ModuleRegistry stores the metadata and factory lambdas, ModuleFactory (defined in base/include/declarative/ModuleFactory.h and implemented in base/src/declarative/ModuleFactory.cpp【/cache/repos/github.com/apra-labs/aprapipes/main/base/src/declarative/ModuleFactory.cpp#L1-L30】) orchestrates the actual instantiation during pipeline construction.

The factory operates through three distinct stages:

1. Pipeline Description Parsing

The command-line interface in base/tools/aprapipes_cli.cpp (lines 94-100) reads JSON or TOML configuration files and resolves module names and property maps into a PipelineDescription structure. This declarative approach separates pipeline topology from implementation details.

2. Metadata Lookup

When constructing a specific module, ModuleFactory::createModule queries ModuleRegistry::instance() for the ModuleInfo associated with the requested string name. This lookup operates in constant time using the registry's internal map, returning the pre-computed metadata and factory lambda.

3. Concrete Instantiation

The factory invokes the appropriate creation method based on module capabilities:

• CPU-only modules: Calls info.factory(props) to execute the stored lambda, returning a std::unique_ptr<Module>.
• CUDA-enabled modules: Detects requiresCudaStream (set during registration in ModuleRegistrations.cpp), invokes createCudaStream() to obtain a thread-local CUDA stream, and calls info.cudaFactory(props, streamPtr).

The created module is stored in an internal TypeFactory<Module*, size_t, ...> keyed by a type ID derived from the module name hash, with deduplication handled via modules_map to ensure that identical instance IDs in the pipeline description yield a single concrete object.

Dynamic Loading in Practice

The ModuleFactory and registration pattern enables several real-world workflows that eliminate compile-time dependencies between pipeline definitions and module implementations.

Instantiating from Declarative Configuration

A typical TOML pipeline description specifies modules by string name:

[modules.reader]
type = "FileReaderModule"
[modules.reader.props]
path = "data/frame.raw"

At runtime, ModuleFactory resolves "FileReaderModule" through the registry and invokes the factory lambda created by REGISTER_MODULE(FileReaderModule, FileReaderModuleProps) in base/src/declarative/ModuleRegistrations.cpp. The pipeline builder never includes a header file for FileReaderModule, maintaining strict separation between module implementation and pipeline orchestration.

Adding New Modules Without Factory Modifications

To extend the framework with a custom transform, implement the module class and register it:

// MyCoolTransform.h
#include "Module.h"

class MyCoolTransform : public apra::Module {
public:
    struct Metadata {
        static constexpr std::string_view name = "MyCoolTransform";
        static constexpr apra::ModuleCategory category = apra::ModuleCategory::Transform;
        static constexpr std::string_view version = "1.0";
        static constexpr std::array<apra::PinDef, 1> inputs = {/* ... */};
        static constexpr std::array<apra::PinDef, 1> outputs = {/* ... */};
        static constexpr std::array<apra::PropDef, 1> properties = {/* ... */};
    };
    
    explicit MyCoolTransform(const MyCoolTransformProps& props) { /* ... */ }
};

// MyCoolTransform.cpp
#include "MyCoolTransform.h"
REGISTER_MODULE(MyCoolTransform, MyCoolTransformProps);

After compiling and linking this translation unit, MyCoolTransform becomes available to any pipeline description referencing type = "MyCoolTransform", with no modifications to ModuleFactory or ModuleRegistry source code.

CUDA-Aware Module Instantiation

The pattern transparently handles CUDA resource management. When ModuleFactory detects that a module's ModuleInfo has requiresCudaStream set to true (populated during registration in ModuleRegistrations.cpp), it automatically calls createCudaStream() to obtain a thread-local CUDA stream before invoking info.cudaFactory(props, streamPtr).

[modules.cuda_encoder]
type = "ImageEncoderCV"
[modules.cuda_encoder.props]
format = "h264"

The pipeline author specifies only the module name and properties; the factory handles CUDA context and stream creation internally, passing the appropriate resources to the module's constructor via the registered cudaFactory lambda.

Why This Pattern Enables Dynamic Loading

The ModuleFactory and registration pattern achieves true dynamic loading through architectural decisions that decouple module implementation from pipeline construction:

Characteristic	Implementation Detail
Zero-runtime registration code	REGISTER_MODULE runs at static initialization, populating a global map before main() executes.
Metadata-driven creation	ModuleInfo stores name, pins, properties, and ready-to-invoke factory lambdas, enabling instantiation by string lookup.
Decoupled builder	ModuleFactory never includes concrete module headers; it queries the registry by name and invokes the stored lambda.
Plug-and-play linking	Adding a new compiled module automatically extends the set of names the factory can handle without code changes.
Thread-local CUDA resources	CUDA stream creation is hidden behind the same factory interface, triggered by requiresCudaStream metadata.
Test isolation	addRegistrationCallback allows test suites to clear the registry and re-register modules for clean state between tests.

Together, these mechanisms give ApraPipes a dynamic, declarative pipeline runtime while maintaining the performance benefits of static C++ typing.

Summary

• The ModuleFactory and registration pattern in ApraPipes enables runtime instantiation of C++ modules from JSON or TOML configuration files.
• The REGISTER_MODULE macro in base/include/declarative/ModuleRegistry.h executes at static initialization, automatically populating the ModuleRegistry with metadata and factory lambdas for every linked module.
• ModuleFactory in base/include/declarative/ModuleFactory.h queries the registry by string name to retrieve ModuleInfo, then invokes the stored factory lambda to create concrete instances without including module-specific headers.
• New modules become available to pipeline descriptions immediately after compilation and linking, with no modifications required to the factory or pipeline construction code.
• CUDA-enabled modules receive automatic thread-local stream allocation through the same registration interface, triggered by requiresCudaStream metadata set during registration.

Frequently Asked Questions

How does the REGISTER_MODULE macro work without explicit runtime calls?

The REGISTER_MODULE macro expands to a static initialization statement that executes during program startup, before main() runs. In base/include/declarative/ModuleRegistry.h【/cache/repos/github.com/apra-labs/aprapipes/main/base/include/declarative/ModuleRegistry.h#L382-L464】, the macro creates an anonymous struct with a constructor that calls ModuleRegistry::instance().registerModule(). When the translation unit loads, the linker ensures this constructor runs, automatically registering the module without any explicit registration function calls in user code.

Can I use ModuleFactory to create modules without knowing their types at compile time?

Yes. ModuleFactory in base/src/declarative/ModuleFactory.cpp【/cache/repos/github.com/apra-labs/aprapipes/main/base/src/declarative/ModuleFactory.cpp#L1-L30】 is designed specifically for type-erased instantiation. You provide a string name (e.g., "FileReaderModule") and a property map, and the factory queries ModuleRegistry to retrieve the pre-stored factory lambda. The lambda creates the concrete type and returns it as a std::unique_ptr<Module>, allowing your pipeline construction code to remain completely decoupled from specific module implementations.

What happens if two modules have the same name in the registry?

The ModuleRegistry implementation in base/include/declarative/ModuleRegistry.h treats module names as unique identifiers. When registerModule() is called, it typically inserts the module into an internal map keyed by the name string. If a duplicate registration occurs, the behavior depends on the specific registry implementation—usually either overwriting the previous entry or asserting/throwing an error. In practice, the REGISTER_MODULE macro uses the class name as the key, and since C++ type names are unique within a program, collisions only occur if two different modules explicitly define the same Metadata::name string, which should be avoided by convention.

How does the pattern handle CUDA-specific module requirements?

CUDA-enabled modules use an extended registration path that captures a cudaFactory lambda alongside the standard factory. In base/src/declarative/ModuleFactory.cpp, when ModuleFactory detects that a module's ModuleInfo has requiresCudaStream set to true (populated during registration in ModuleRegistrations.cpp), it automatically calls createCudaStream() to obtain a thread-local CUDA stream before invoking info.cudaFactory(props, streamPtr). This allows the pipeline description to remain agnostic to whether modules use CPU or GPU processing, while ensuring that CUDA resources are properly managed and passed to module constructors that require them.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}