How Modly Loads AI Models at Startup: The Generator Registry Discovery Process

At startup, Modly's GeneratorRegistry discovers, validates, and instantiates every AI model extension by scanning the extensions directory, choosing between direct import or subprocess isolation, and finally activating the model specified in the SELECTED_MODEL_ID environment variable.

When the Modly application launches, the system must prepare all available AI generators before handling inference requests. The lightningpixel/modly repository implements a centralized GeneratorRegistry that orchestrates this loading sequence, determining whether to run models in-process or inside isolated subprocesses based on the presence of manifest.json and generator.py files.

The GeneratorRegistry Initialization Sequence

The loading process begins when the singleton generator_registry is created at import time in api/services/generator_registry.py. Its constructor, GeneratorRegistry.__init__, initializes empty internal maps for storing generators and reads the SELECTED_MODEL_ID environment variable, defaulting to sf3d if unspecified. This singleton instance remains the single point of contact for all model management throughout the application lifecycle.

Extension Discovery and Validation

Scanning the Extensions Directory

The registry's initialize() method triggers the internal helper _discover_extensions(), which walks the directory specified by the EXTENSIONS_DIR environment variable. This function inspects every subdirectory, searching specifically for a manifest.json file alongside a generator.py file. The registry recognizes these two files as the minimal requirements for any valid Modly model extension.

Manifest Validation and Filtering

During the scan, the registry validates each discovered extension by checking the type field within manifest.json. If a folder lacks either required file, or if the manifest's type value is not "model", the extension is skipped with a logged warning. Only extensions passing this validation proceed to the instantiation phase.

Direct Mode vs. Subprocess Mode Loading

Modly supports two distinct loading strategies determined by the presence of a virtual environment within the extension folder. In direct mode (legacy), used when no venv exists, the registry uses importlib.util to build and import the module directly from generator.py, then extracts the class named in manifest["generator_class"]. This approach runs the AI model inside the main Modly process.

In subprocess mode, engaged when a virtual environment is detected or when build_vendor.py requires execution, the registry wraps the extension in an ExtensionProcess instance from api/services/extension_process.py. This runs the generator inside an isolated subprocess to prevent dependency conflicts between different AI models.

Generator Instantiation and Registration

Once discovery completes, the registry iterates over validated entries to create concrete generator instances. For subprocess models, it instantiates ExtensionProcess and configures the model_dir and outputs_dir paths. For direct-mode models, it calls the generator class constructor with MODELS_DIR / model_id and WORKSPACE_DIR as arguments, attaching additional manifest fields such as hf_repo and params_schema directly to the instance.

The registry stores each generator object in _generators[model_id] and its manifest in _manifests, clearing any previous error states for that model. Extensions declaring multiple nodes receive individual entries formatted as ext_id/node_id in the internal maps, allowing granular access to specific capabilities within a single extension package.

Activating the Default AI Model

After populating the registry, the system verifies that the requested SELECTED_MODEL_ID exists among the loaded generators. If the specified ID is missing, the registry automatically falls back to the first discovered model and updates its internal _active_id accordingly. The first subsequent call to get_active() triggers lazy downloading and loading of the selected generator if model weights are not already present locally.

Working with the Generator Registry at Runtime

Developers can interact with the registry programmatically to reload models or switch contexts during execution without restarting the application.

Triggering a registry reload after installing a new model:

from services.generator_registry import generator_registry

# Re-scan the extensions folder and rebuild the registry

generator_registry.reload()

Switching the active model at runtime:

from services.generator_registry import generator_registry

# Change the active model to the one with ID "my-model"

generator_registry.switch_model("my-model")
active_gen = generator_registry.get_active()

Accessing a generator's parameter schema:

from services.generator_registry import generator_registry

schema = generator_registry.params_schema("my-model")
print(schema)   # → list of dicts describing the model's configurable arguments

Running inference with the active generator:

from services.generator_registry import generator_registry

gen = generator_registry.get_active()

# Ensure the model is downloaded and loaded

gen = generator_registry.get_active()   # lazy download/load happens here

# Call the generator's generate method (signature defined in BaseGenerator)

result = gen.generate(input_image, **gen.default_params())

Summary

• The GeneratorRegistry in api/services/generator_registry.py serves as the central authority for AI model lifecycle management in Modly.
• At startup, _discover_extensions() scans EXTENSIONS_DIR for folders containing both manifest.json and generator.py.
• Extensions load either in direct mode (in-process via importlib) or subprocess mode (isolated via ExtensionProcess), depending on virtual environment presence.
• The registry instantiates each validated generator, stores it in _generators, and selects the active model based on SELECTED_MODEL_ID with automatic fallback to the first available model.
• Runtime methods like switch_model(), reload(), and get_active() enable dynamic model management, with lazy loading triggered on first access.

Frequently Asked Questions

What is the GeneratorRegistry in Modly?

The GeneratorRegistry is a singleton class defined in api/services/generator_registry.py that discovers, instantiates, and manages AI model extensions during Modly startup. It maintains internal dictionaries of generator instances and their manifests, providing a unified interface for accessing models and enforcing the interface contracts defined in api/services/generators/base.py.

How does Modly handle missing or invalid model extensions?

During the discovery phase in _discover_extensions(), the registry skips any directory lacking both manifest.json and generator.py, or where the manifest's type field is not "model". These invalid extensions are ignored with a warning log entry, allowing the startup process to continue loading only valid model definitions.

What is the difference between direct mode and subprocess mode?

Direct mode imports and runs the generator class directly in the main Modly process using importlib.util, suitable for simple extensions without conflicting dependencies. Subprocess mode wraps the extension in an ExtensionProcess object from api/services/extension_process.py, running it inside an isolated subprocess with its own virtual environment to prevent dependency conflicts between different AI models.

How can I programmatically switch AI models at runtime?

Import the generator_registry singleton from api/services/generator_registry.py and call the switch_model(model_id) method with the desired model identifier. After switching, call get_active() to retrieve the new generator instance. The registry handles lazy loading automatically, downloading model weights from the hf_repo specified in the manifest if necessary when first accessing the generator.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}