# How Modly Loads AI Models at Startup: The Generator Registry Discovery Process

> Learn how Modly's GeneratorRegistry loads AI models at startup by scanning extensions, choosing import methods, and activating the selected model. Discover the Modly startup process.

- Repository: [lightningpixel/modly](https://github.com/lightningpixel/modly)
- Tags: internals
- Published: 2026-08-15

---

**At startup, Modly's GeneratorRegistry discovers, validates, and instantiates every AI model extension by scanning the extensions directory, choosing between direct import or subprocess isolation, and finally activating the model specified in the `SELECTED_MODEL_ID` environment variable.**

When the Modly application launches, the system must prepare all available AI generators before handling inference requests. The `lightningpixel/modly` repository implements a centralized **GeneratorRegistry** that orchestrates this loading sequence, determining whether to run models in-process or inside isolated subprocesses based on the presence of [`manifest.json`](https://github.com/lightningpixel/modly/blob/main/manifest.json) and [`generator.py`](https://github.com/lightningpixel/modly/blob/main/generator.py) files.

## The GeneratorRegistry Initialization Sequence

The loading process begins when the singleton `generator_registry` is created at import time in [`api/services/generator_registry.py`](https://github.com/lightningpixel/modly/blob/main/api/services/generator_registry.py). Its constructor, `GeneratorRegistry.__init__`, initializes empty internal maps for storing generators and reads the `SELECTED_MODEL_ID` environment variable, defaulting to `sf3d` if unspecified. This singleton instance remains the single point of contact for all model management throughout the application lifecycle.

## Extension Discovery and Validation

### Scanning the Extensions Directory

The registry's `initialize()` method triggers the internal helper `_discover_extensions()`, which walks the directory specified by the `EXTENSIONS_DIR` environment variable. This function inspects every subdirectory, searching specifically for a [`manifest.json`](https://github.com/lightningpixel/modly/blob/main/manifest.json) file alongside a [`generator.py`](https://github.com/lightningpixel/modly/blob/main/generator.py) file. The registry recognizes these two files as the minimal requirements for any valid Modly model extension.

### Manifest Validation and Filtering

During the scan, the registry validates each discovered extension by checking the `type` field within [`manifest.json`](https://github.com/lightningpixel/modly/blob/main/manifest.json). If a folder lacks either required file, or if the manifest's `type` value is not `"model"`, the extension is skipped with a logged warning. Only extensions passing this validation proceed to the instantiation phase.

## Direct Mode vs. Subprocess Mode Loading

Modly supports two distinct loading strategies determined by the presence of a virtual environment within the extension folder. In **direct mode** (legacy), used when no `venv` exists, the registry uses `importlib.util` to build and import the module directly from [`generator.py`](https://github.com/lightningpixel/modly/blob/main/generator.py), then extracts the class named in `manifest["generator_class"]`. This approach runs the AI model inside the main Modly process.

In **subprocess mode**, engaged when a virtual environment is detected or when [`build_vendor.py`](https://github.com/lightningpixel/modly/blob/main/build_vendor.py) requires execution, the registry wraps the extension in an `ExtensionProcess` instance from [`api/services/extension_process.py`](https://github.com/lightningpixel/modly/blob/main/api/services/extension_process.py). This runs the generator inside an isolated subprocess to prevent dependency conflicts between different AI models.

## Generator Instantiation and Registration

Once discovery completes, the registry iterates over validated entries to create concrete generator instances. For subprocess models, it instantiates `ExtensionProcess` and configures the `model_dir` and `outputs_dir` paths. For direct-mode models, it calls the generator class constructor with `MODELS_DIR / model_id` and `WORKSPACE_DIR` as arguments, attaching additional manifest fields such as `hf_repo` and `params_schema` directly to the instance.

The registry stores each generator object in `_generators[model_id]` and its manifest in `_manifests`, clearing any previous error states for that model. Extensions declaring multiple nodes receive individual entries formatted as `ext_id/node_id` in the internal maps, allowing granular access to specific capabilities within a single extension package.

## Activating the Default AI Model

After populating the registry, the system verifies that the requested `SELECTED_MODEL_ID` exists among the loaded generators. If the specified ID is missing, the registry automatically falls back to the first discovered model and updates its internal `_active_id` accordingly. The first subsequent call to `get_active()` triggers lazy downloading and loading of the selected generator if model weights are not already present locally.

## Working with the Generator Registry at Runtime

Developers can interact with the registry programmatically to reload models or switch contexts during execution without restarting the application.

Triggering a registry reload after installing a new model:

```python
from services.generator_registry import generator_registry

# Re-scan the extensions folder and rebuild the registry

generator_registry.reload()

```

Switching the active model at runtime:

```python
from services.generator_registry import generator_registry

# Change the active model to the one with ID "my-model"

generator_registry.switch_model("my-model")
active_gen = generator_registry.get_active()

```

Accessing a generator's parameter schema:

```python
from services.generator_registry import generator_registry

schema = generator_registry.params_schema("my-model")
print(schema)   # → list of dicts describing the model's configurable arguments

```

Running inference with the active generator:

```python
from services.generator_registry import generator_registry

gen = generator_registry.get_active()

# Ensure the model is downloaded and loaded

gen = generator_registry.get_active()   # lazy download/load happens here

# Call the generator's generate method (signature defined in BaseGenerator)

result = gen.generate(input_image, **gen.default_params())

```

## Summary

- The **GeneratorRegistry** in [`api/services/generator_registry.py`](https://github.com/lightningpixel/modly/blob/main/api/services/generator_registry.py) serves as the central authority for AI model lifecycle management in Modly.
- At startup, `_discover_extensions()` scans `EXTENSIONS_DIR` for folders containing both [`manifest.json`](https://github.com/lightningpixel/modly/blob/main/manifest.json) and [`generator.py`](https://github.com/lightningpixel/modly/blob/main/generator.py).
- Extensions load either in **direct mode** (in-process via `importlib`) or **subprocess mode** (isolated via `ExtensionProcess`), depending on virtual environment presence.
- The registry instantiates each validated generator, stores it in `_generators`, and selects the active model based on `SELECTED_MODEL_ID` with automatic fallback to the first available model.
- Runtime methods like `switch_model()`, `reload()`, and `get_active()` enable dynamic model management, with lazy loading triggered on first access.

## Frequently Asked Questions

### What is the GeneratorRegistry in Modly?

The GeneratorRegistry is a singleton class defined in [`api/services/generator_registry.py`](https://github.com/lightningpixel/modly/blob/main/api/services/generator_registry.py) that discovers, instantiates, and manages AI model extensions during Modly startup. It maintains internal dictionaries of generator instances and their manifests, providing a unified interface for accessing models and enforcing the interface contracts defined in [`api/services/generators/base.py`](https://github.com/lightningpixel/modly/blob/main/api/services/generators/base.py).

### How does Modly handle missing or invalid model extensions?

During the discovery phase in `_discover_extensions()`, the registry skips any directory lacking both [`manifest.json`](https://github.com/lightningpixel/modly/blob/main/manifest.json) and [`generator.py`](https://github.com/lightningpixel/modly/blob/main/generator.py), or where the manifest's `type` field is not `"model"`. These invalid extensions are ignored with a warning log entry, allowing the startup process to continue loading only valid model definitions.

### What is the difference between direct mode and subprocess mode?

**Direct mode** imports and runs the generator class directly in the main Modly process using `importlib.util`, suitable for simple extensions without conflicting dependencies. **Subprocess mode** wraps the extension in an `ExtensionProcess` object from [`api/services/extension_process.py`](https://github.com/lightningpixel/modly/blob/main/api/services/extension_process.py), running it inside an isolated subprocess with its own virtual environment to prevent dependency conflicts between different AI models.

### How can I programmatically switch AI models at runtime?

Import the `generator_registry` singleton from [`api/services/generator_registry.py`](https://github.com/lightningpixel/modly/blob/main/api/services/generator_registry.py) and call the `switch_model(model_id)` method with the desired model identifier. After switching, call `get_active()` to retrieve the new generator instance. The registry handles lazy loading automatically, downloading model weights from the `hf_repo` specified in the manifest if necessary when first accessing the generator.