How to Implement a GeneratorRegistry with Model Adapters in FastAPI

Modly implements a dynamic GeneratorRegistry that discovers, loads, and manages 3-D model adapters at runtime using a plugin architecture that eliminates the need to modify core FastAPI server code when adding new AI generation capabilities.

Modly is an open-source FastAPI application that generates 3-D assets through pluggable AI models. The system centers on a GeneratorRegistry that dynamically discovers model adapters from the filesystem and exposes a unified lifecycle management API for seamless integration with HTTP endpoints.

Understanding the Registry Architecture

The GeneratorRegistry class defined in api/services/generator_registry.py serves as a singleton orchestrator that maintains internal mappings of extension IDs to concrete generator instances. It handles the complete lifecycle of model adapters, from initial discovery through memory management and runtime switching, supporting both in-process and isolated subprocess execution modes.

The registry initializes during FastAPI startup through generator_registry.initialize(), which triggers the discovery mechanism and sets the default active model based on the SELECTED_MODEL_ID environment variable.

The BaseGenerator Contract

All model adapters must inherit from BaseGenerator, located in api/services/generators/base.py. This abstract base class establishes the required interface ensuring compatibility with the registry and FastAPI endpoints.

Required methods include:

• load(): Loads model weights into GPU or CPU memory.
• unload(): Releases memory resources and performs custom cleanup.
• generate(image_bytes, params, progress_cb, cancel_event): Executes 3-D generation logic and returns a Path to the resulting .glb file.
• is_downloaded() and _auto_download(): Utility methods for managing model file downloads from Hugging Face repositories.
• params_schema(): Returns the UI configuration schema injected from the manifest.

Extension Discovery and Loading

Directory Scanning Mechanism

When initialize() runs, the registry executes _discover_extensions() to scan the directory specified by the EXTENSIONS_DIR environment variable. For every subfolder containing both a manifest.json and a generator.py, the registry constructs a full identifier using the pattern ext_id/node_id and prepares the adapter for loading.

Direct vs. Subprocess Loading Modes

The discovery logic supports two distinct loading strategies based on extension structure:

• Direct (legacy): The module is imported using importlib and instantiated directly within the main FastAPI process.
• Subprocess (isolated): If the extension includes a virtual environment (venv) or a build_vendor.py script, the registry creates an ExtensionProcess wrapper from api/services/extension_process.py that runs the model in its own process to prevent dependency conflicts.

Registry API for FastAPI Integration

The registry exposes several public methods that FastAPI routers consume to manage model state dynamically:

• get_active(): Returns the currently selected generator, automatically triggering download and loading via _auto_download() if the model isn't already in memory.
• switch_model(model_id): Unloads the previously active model and initializes the requested adapter.
• reload(): Re-scans the extensions directory and rebuilds internal mappings without requiring a server restart.
• active_status() and all_status(): Generate JSON payloads consumed by monitoring endpoints to report current model state and availability.

Implementing a Custom Model Adapter

To add a new generation model, create a directory under EXTENSIONS_DIR containing a manifest and implementation file.

First, define the manifest in extensions/my_adapter/manifest.json:

{
  "id": "my_adapter",
  "name": "My Awesome Model",
  "generator_class": "MyGenerator",
  "hf_repo": "myorg/my-model",
  "vram_gb": 8,
  "nodes": [
    {
      "id": "default",
      "name": "Default Node",
      "params_schema": []
    }
  ]
}

Next, implement the adapter in extensions/my_adapter/generator.py:

from services.generators.base import BaseGenerator
from pathlib import Path

class MyGenerator(BaseGenerator):
    MODEL_ID = "my_adapter"
    DISPLAY_NAME = "My Awesome Model"
    VRAM_GB = 8

    def load(self) -> None:
        # Insert model loading logic here (e.g., torch.load)

        self._model = "model object"

    def generate(self, image_bytes, params, progress_cb=None, cancel_event=None):
        # Example: pretend we always return the same mesh

        output_path = self.outputs_dir / "result.glb"
        with open(output_path, "wb") as f:
            f.write(b"glb-binary-data")
        return output_path

Managing Model Lifecycles at Runtime

The registry enables hot-swapping of models without server restarts. When SELECTED_MODEL_ID is set, the registry automatically activates that model during initialization. To change models dynamically:

curl -X POST "http://localhost:8000/model/switch" -d "model_id=my_adapter"

To pick up newly added extensions from the filesystem without restarting FastAPI:

from services.generator_registry import generator_registry
generator_registry.reload()

Integrating with FastAPI Routers

The api/routers/model.py file maps registry methods to HTTP endpoints:

Endpoint	Registry Method	Purpose
GET /model/status	active_status()	Returns metadata about the active model
GET /model/all	all_status()	Lists all discovered adapters and their states
GET /model/params	params_schema()	Retrieves the parameter schema for UI rendering
POST /model/switch	switch_model()	Changes the active generator
POST /model/unload-all	unload_all()	Frees all model memory from VRAM

When handling generation requests, retrieve the active generator through the registry:

# api/routers/generation.py

@router.post("/generate")
async def generate(request: GenerationRequest):
    gen = generator_registry.get_active()
    result_path = gen.generate(
        image_bytes=request.image,
        params=request.params,
        progress_cb=lambda pct, step: print(f"{pct}% – {step}"),
    )
    return {"glb_path": str(result_path)}

Summary

• The GeneratorRegistry in api/services/generator_registry.py provides centralized discovery and lifecycle management for model adapters without requiring core code modifications.
• Adapters must implement the BaseGenerator interface from api/services/generators/base.py, specifically the load(), unload(), and generate() methods.
• Discovery scans the EXTENSIONS_DIR for folders containing manifest.json and generator.py, supporting both direct import and isolated subprocess execution via ExtensionProcess.
• The registry exposes methods like get_active(), switch_model(), and reload() that integrate directly with FastAPI endpoints defined in api/routers/model.py.
• New adapters require only a manifest file and a Python class extending BaseGenerator, enabling dynamic loading determined by the SELECTED_MODEL_ID environment variable.

Frequently Asked Questions

How does the GeneratorRegistry discover new model adapters?

The registry scans the directory specified by the EXTENSIONS_DIR environment variable during initialization. It identifies valid adapters by locating folders containing both a manifest.json configuration file and a generator.py implementation file. The _discover_extensions() method in api/services/generator_registry.py constructs full identifiers using the pattern ext_id/node_id for each valid extension found.

Can I run model adapters in isolated environments?

Yes. The registry supports subprocess isolation for adapters that ship with their own virtual environment (venv) or a build_vendor.py script. When these conditions are detected, the registry creates an ExtensionProcess wrapper that runs the model in a separate process, preventing dependency conflicts with the core FastAPI application.

What methods must a custom model adapter implement?

Every adapter must extend BaseGenerator and implement three core methods: load() to initialize model weights into memory, unload() to free GPU/VRAM resources, and generate(image_bytes, params, ...) to process inputs and return a file path. Optional methods like is_downloaded() and _auto_download() enable automatic model file management from Hugging Face repositories specified in the manifest.

How do I switch between models without restarting the server?

Use the switch_model(model_id) method exposed by the registry, accessible via the POST /model/switch endpoint. This method unloads the current adapter from memory, initializes the requested model, and updates the active status. You can also call reload() to pick up newly added extensions from the filesystem without restarting the FastAPI server.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}