# How Modly's API Layer Is Structured and How It Interfaces With the FastAPI Backend

> Explore Modly's API layer architecture, including its FastAPI backend, modular routers, GeneratorRegistry, and BaseGenerator subclasses for efficient model inference.

- Repository: [lightningpixel/modly](https://github.com/lightningpixel/modly)
- Tags: architecture
- Published: 2026-08-20

---

**Modly's API layer uses a FastAPI application with modular routers, a central `GeneratorRegistry` for model discovery, and abstract `BaseGenerator` subclasses that handle inference either in-process or via isolated subprocesses.**

Modly is an open-source AI inference framework for 3D generation. Its **API layer architecture** cleanly separates HTTP concerns from model-specific logic, enabling extensibility through a plugin-based extension system. This article examines the core components and their interactions based on the actual source code in the `lightningpixel/modly` repository.

## Core Components of the API Layer

### [`api/main.py`](https://github.com/lightningpixel/modly/blob/main/api/main.py): FastAPI Application Entry Point

The root of the API layer resides in [`api/main.py`](https://github.com/lightningpixel/modly/blob/main/api/main.py). This file instantiates the FastAPI app, configures **CORS middleware**, registers all routers, and defines an **asynchronous lifespan context** that manages the full application lifecycle.

```python

# api/main.py

app = FastAPI(
    title="Modly API",
    version="0.4.1",
    lifespan=lifespan,
)

```

The `lifespan` context manager is critical: it calls `GeneratorRegistry.initialize()` on startup to discover and load extensions, then performs graceful shutdown when the application terminates.

Routers are included with optional URL prefixes:

```python
app.include_router(generation.router, prefix="/generate")
app.include_router(status.router)

# … additional routers …

```

The same file also implements `serve_workspace_file`, which streams generated assets from `WORKSPACE_DIR` back to clients.

### Modular Routers in `api/routers/`

Endpoints are organized by domain into separate `APIRouter` modules:

| Router File | Purpose |
|-------------|---------|
| [`api/routers/generation.py`](https://github.com/lightningpixel/modly/blob/main/api/routers/generation.py) | Image-to-mesh generation, job tracking, cancellation, progress callbacks |
| [`api/routers/status.py`](https://github.com/lightningpixel/modly/blob/main/api/routers/status.py) | Health checks for the Electron frontend |
| [`api/routers/models.py`](https://github.com/lightningpixel/modly/blob/main/api/routers/models.py) | Model management operations |
| [`api/routers/settings.py`](https://github.com/lightningpixel/modly/blob/main/api/routers/settings.py) | Configuration endpoints |
| [`api/routers/extensions.py`](https://github.com/lightningpixel/modly/blob/main/api/routers/extensions.py) | Extension discovery and metadata |
| [`api/routers/export.py`](https://github.com/lightningpixel/modly/blob/main/api/routers/export.py) | Export format handling |
| [`api/routers/workflow_runs.py`](https://github.com/lightningpixel/modly/blob/main/api/routers/workflow_runs.py) | Workflow execution |
| [`api/routers/agent.py`](https://github.com/lightningpixel/modly/blob/main/api/routers/agent.py) | Agent-related operations |

Each router encapsulates related functionality and is imported into [`api/main.py`](https://github.com/lightningpixel/modly/blob/main/api/main.py) for registration.

## Generator Registry: The Bridge to AI Inference

### `services/generator_registry.GeneratorRegistry`

Located in [`api/services/generator_registry.py`](https://github.com/lightningpixel/modly/blob/main/api/services/generator_registry.py), the `GeneratorRegistry` is the **central coordination point** between the HTTP layer and model implementations. It performs three core responsibilities:

- **Discovery**: Scans `EXTENSIONS_DIR` for valid extensions (folders containing [`manifest.json`](https://github.com/lightningpixel/modly/blob/main/manifest.json))
- **Instantiation**: Creates either direct `BaseGenerator` instances or `ExtensionProcess` wrappers based on extension configuration
- **Lifecycle management**: Handles model switching, loading, unloading, and status tracking

Environment variables configure critical paths (lines 24-31 in [`generator_registry.py`](https://github.com/lightningpixel/modly/blob/main/generator_registry.py)):

```python
EXTENSIONS_DIR  # Where user extensions are located

WORKSPACE_DIR   # Output directory for generated files

MODELS_DIR      # Cache for downloaded model checkpoints

```

### `services.generators.base.BaseGenerator`

All model adapters must inherit from the abstract base class defined in [`api/services/generators/base.py`](https://github.com/lightningpixel/modly/blob/main/api/services/generators/base.py). The contract requires implementation of:

- `generate()` – Execute inference and write output to workspace
- `load()` – Initialize model weights and move to target device
- `unload()` – Free GPU memory and resources
- `is_loaded()` – Check load state
- `is_downloaded()` – Verify model weights are cached locally

### `services.extension_process.ExtensionProcess`

For extensions shipping with their own **virtual environment**, [`api/services/extension_process.py`](https://github.com/lightningpixel/modly/blob/main/api/services/extension_process.py) provides a subprocess wrapper that implements the same public interface as `BaseGenerator`. This allows the rest of the API to treat both execution modes uniformly—whether running in-process or isolated.

## Request-Response Flow: End-to-End Interaction

The **interface between FastAPI backend and model inference** follows a structured pipeline:

1. **Application startup** – FastAPI invokes the `lifespan` context; `GeneratorRegistry.initialize()` discovers extensions and prepares the workspace

2. **Request routing** – HTTP requests route to appropriate `APIRouter` endpoints; for example, `POST /generate/from-image` hits [`api/routers/generation.py`](https://github.com/lightningpixel/modly/blob/main/api/routers/generation.py)

3. **Model selection** – The endpoint validates `model_id` and calls `generator_registry.switch_model(model_id)` to activate the requested generator

4. **Model loading** – `generator_registry.get_active()` triggers on-demand download (if needed) and loads the model in a **background thread** with progress callbacks—keeping the route non-blocking

5. **Generation execution** – A background task (`_run_generation`) invokes the generator's `generate` method with image bytes, parameters, progress callback, and optional cancellation event; output writes to `WORKSPACE_DIR/<collection>`

6. **Result serving** – The endpoint returns a job ID for polling; completed results are accessible via `/workspace/<relative-path>` served by `serve_workspace_file`

## Adding Custom Model Extensions

The architecture enables **hot-pluggable models** without modifying core API code:

1. Create extension folder: `/path/to/extensions/my_model/`

2. Add [`manifest.json`](https://github.com/lightningpixel/modly/blob/main/manifest.json):

```json
{
  "id": "my_model",
  "name": "My Model",
  "generator_class": "MyGenerator",
  "type": "model",
  "nodes": []
}

```

3. Implement [`generator.py`](https://github.com/lightningpixel/modly/blob/main/generator.py) subclassing `BaseGenerator`

4. Set `EXTENSIONS_DIR=/path/to/extensions` and restart Modly

The `GeneratorRegistry` automatically discovers and registers the extension on startup.

## Calling the Generation API

```http
POST /generate/from-image HTTP/1.1
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary

------WebKitFormBoundary
Content-Disposition: form-data; name="image"; filename="example.png"
Content-Type: image/png

<binary PNG data>
------WebKitFormBoundary
Content-Disposition: form-data; name="model_id"

sf3d
------WebKitFormBoundary
Content-Disposition: form-data; name="params"

{"prompt":"A futuristic cityscape"}
------WebKitFormBoundary--

```

The response contains a `job_id`. Poll `GET /generate/status/{job_id}` until status is `"done"`, then retrieve the result at the provided `output_url`.

## Summary

- **[`api/main.py`](https://github.com/lightningpixel/modly/blob/main/api/main.py)** instantiates FastAPI, configures middleware, registers routers, and manages application lifespan
- **`APIRouter` modules** in `api/routers/` group endpoints by domain with optional URL prefixes
- **`GeneratorRegistry`** discovers extensions, manages model lifecycle, and abstracts execution mode (direct or subprocess)
- **`BaseGenerator`** defines the contract all model adapters must implement
- **`ExtensionProcess`** provides isolation for extensions with custom virtual environments
- **Background task execution** keeps HTTP routes responsive during model loading and generation
- **Plugin-based extension system** allows adding new models by dropping folders into `EXTENSIONS_DIR`

## Frequently Asked Questions

### What file serves as the FastAPI application entry point in Modly?

[`api/main.py`](https://github.com/lightningpixel/modly/blob/main/api/main.py) serves as the entry point. It creates the FastAPI instance with `lifespan` context management, adds CORS middleware, includes all routers with their prefixes, and implements workspace file serving.

### How does Modly isolate extensions that require different Python dependencies?

Modly uses `ExtensionProcess` in [`api/services/extension_process.py`](https://github.com/lightningpixel/modly/blob/main/api/services/extension_process.py) to wrap such extensions. This class runs the generator in a separate subprocess with its own virtual environment while exposing the same interface as direct `BaseGenerator` instances, allowing uniform treatment by the registry and routers.

### Can new AI models be added without modifying Modly's source code?

Yes. Place a folder containing [`manifest.json`](https://github.com/lightningpixel/modly/blob/main/manifest.json) and a [`generator.py`](https://github.com/lightningpixel/modly/blob/main/generator.py) implementing `BaseGenerator` into the directory specified by `EXTENSIONS_DIR`. The `GeneratorRegistry` automatically discovers and loads it on the next startup.

### How does Modly prevent model loading from blocking HTTP requests?

Both model loading and generation execute in **background threads** managed through FastAPI's `BackgroundTask` or `asyncio` mechanisms. Progress callbacks update job status while the initial HTTP response returns immediately with a job ID for polling.