# Modly API Structure and Backend Architecture Explained

> Explore Modly's backend architecture. Discover its modular REST API, dynamic GeneratorRegistry, and FastAPI server embedded in the Electron client. Learn how Modly manages AI model extensions.

- Repository: [lightningpixel/modly](https://github.com/lightningpixel/modly)
- Tags: architecture
- Published: 2026-08-19

---

**Modly's backend is a FastAPI server embedded within the Electron client, centered on a GeneratorRegistry that dynamically discovers and manages AI model extensions through a modular REST API organized under the `api/routers/` package.**

The lightningpixel/modly repository implements a lightweight yet extensible backend architecture that powers its AI-driven 3D generation capabilities. At its core lies a FastAPI application defined in [`api/main.py`](https://github.com/lightningpixel/modly/blob/main/api/main.py) that exposes REST endpoints for model management, generation tasks, and system health, all while running locally inside the Electron desktop application. This design decouples the AI generation logic into swappable extensions while providing a standardized HTTP interface for the frontend to consume.

## Core Backend Components

### FastAPI Application Entry Point (api/main.py)

The backend lifecycle begins in [`api/main.py`](https://github.com/lightningpixel/modly/blob/main/api/main.py), which instantiates the central `FastAPI` application, configures **CORS middleware** for cross-origin requests, and mounts the workspace directory for serving generated assets. This file aggregates all modular routers from the `api/routers/` package and defines the server's startup behavior, creating a unified HTTP interface on a local port that the Electron renderer communicates with via the preload script.

### Router Organization (api/routers/)

Functional endpoints are grouped by domain across separate router modules under `api/routers/`. Each router handles a specific concern: [`generation.py`](https://github.com/lightningpixel/modly/blob/main/generation.py) manages job submission and status polling, [`model.py`](https://github.com/lightningpixel/modly/blob/main/model.py) handles model switching and downloads, [`status.py`](https://github.com/lightningpixel/modly/blob/main/status.py) provides health checks, and additional routers like [`agent.py`](https://github.com/lightningpixel/modly/blob/main/agent.py) support workflow automation. This separation of concerns allows the FastAPI layer to focus strictly on HTTP validation and request routing while delegating stateful operations to underlying services.

### GeneratorRegistry Service (api/services/generator_registry.py)

The architectural heart of the system resides in [`api/services/generator_registry.py`](https://github.com/lightningpixel/modly/blob/main/api/services/generator_registry.py), which implements the **`GeneratorRegistry`** class. This service discovers extension modules within the `EXTENSIONS_DIR`, dynamically loads them either directly in-process or via subprocess isolation, and maintains the active model state. It tracks downloaded models, manages the **`BaseGenerator`** interface for all AI adapters, and provides thread-safe methods like `switch_model()` and `get_active()` that routers invoke to execute generation tasks.

## Request Flow Architecture

When a user submits a generation request, the flow traverses three distinct layers. First, the client POSTs to `/generate/from-image` defined in [`api/routers/generation.py`](https://github.com/lightningpixel/modly/blob/main/api/routers/generation.py), which validates the payload and calls `generator_registry.switch_model()` to activate the requested AI extension.

Next, the endpoint schedules `_run_generation()` as a background task, which retrieves the active generator via `generator_registry.get_active()`, ensures the model weights are downloaded to `~/.modly/models`, and invokes the generator's `generate()` method. Finally, the generator writes outputs to a subdirectory of `WORKSPACE_DIR` (defaulting to `~/.modly/workspace`), and the router updates an in-memory `_jobs` dictionary with status and output URLs accessible via `/generate/status/{job_id}`.

## Key REST Endpoints and Usage Examples

The Modly API exposes standard HTTP endpoints for health monitoring, model management, and asynchronous generation.

### Check System Health

```bash
curl -s http://localhost:PORT/health

# → {"status":"ok"}

```

### List Available Models

```bash
curl -s http://localhost:PORT/model/all

# → [{"id":"sf3d","name":"ShapeFrom3D", "downloaded":true, "loaded":true, ...}, ...]

```

### Switch Active Model

```bash
curl -X POST -H "Content-Type: application/json" \
     -d '{"model_id":"my‑custom‑model"}' \
     http://localhost:PORT/model/switch

# → {"active":"my‑custom‑model"}

```

### Submit Image-to-Mesh Generation Job

```bash
curl -X POST -F image=@input.png \
     -F model_id=sf3d \
     -F collection=MyProject \
     -F remesh=quad \
     -F enable_texture=true \
     -F texture_resolution=1024 \
     http://localhost:PORT/generate/from-image

# → {"job_id":"c8f2a9d4‑…"}

```

### Poll Job Status

```bash
curl http://localhost:PORT/generate/status/<job_id>

# Example response:

# {

#   "job_id":"c8f2a9d4‑…",

#   "status":"done",

#   "progress":100,

#   "output_url":"/workspace/MyProject/mesh.obj"

# }

```

### Stream Model Downloads (SSE)

```bash
curl -N http://localhost:PORT/model/hf-download?repo_id=owner/repo&model_id=sf3d

# Receives lines like: data: {"percent":45,"file":"model.bin","status":"Downloading…"}

```

## Configuration and Storage

The backend configures two primary filesystem locations defined in [`api/services/generator_registry.py`](https://github.com/lightningpixel/modly/blob/main/api/services/generator_registry.py) (lines 24-30). The **`WORKSPACE_DIR`** (default `~/.modly/workspace`) stores generated assets and project outputs, while separate model weights reside in `~/.modly/models`. These paths are configurable and exposed through the FastAPI static file middleware mounted in [`api/main.py`](https://github.com/lightningpixel/modly/blob/main/api/main.py), allowing the frontend to retrieve generated meshes and textures via simple HTTP GET requests to `/workspace/<path>`.

## Summary

- Modly's backend utilizes a **FastAPI server** running locally within the Electron client, initialized in [`api/main.py`](https://github.com/lightningpixel/modly/blob/main/api/main.py) with CORS and router registration.
- **Router modules** under `api/routers/` organize endpoints by function (generation, model, status), keeping HTTP concerns separated from business logic.
- The **GeneratorRegistry** in [`api/services/generator_registry.py`](https://github.com/lightningpixel/modly/blob/main/api/services/generator_registry.py) dynamically discovers, loads, and manages AI model extensions through a consistent `BaseGenerator` interface.
- **Asynchronous generation** uses FastAPI background tasks with in-memory job tracking in `_jobs`, writing outputs to configurable workspace directories.
- The architecture supports **dynamic extension loading**, allowing new AI models to be added by placing them in the `EXTENSIONS_DIR` without modifying core API code.

## Frequently Asked Questions

### What framework powers the Modly backend?

The Modly backend is built on **FastAPI**, a modern Python web framework. The application initializes in [`api/main.py`](https://github.com/lightningpixel/modly/blob/main/api/main.py) with CORS middleware and router registration, providing a high-performance asynchronous server that handles REST requests from the Electron frontend through a local HTTP port.

### How does Modly manage different AI models dynamically?

Modly uses the **GeneratorRegistry** class ([`api/services/generator_registry.py`](https://github.com/lightningpixel/modly/blob/main/api/services/generator_registry.py)) to scan the `EXTENSIONS_DIR` for available extensions, instantiate their `BaseGenerator` implementations, and toggle between them via the `switch_model()` method. This registry supports both in-process and subprocess loading modes for isolation and resource management.

### Where does Modly store generated files and model weights?

By default, Modly stores generated assets in `~/.modly/workspace` and downloaded model weights in `~/.modly/models`, as defined in the generator registry configuration. The FastAPI app mounts the workspace directory as a static file route in [`api/main.py`](https://github.com/lightningpixel/modly/blob/main/api/main.py), enabling direct URL access to outputs like `/workspace/MyProject/mesh.obj`.

### How do I check if the Modly API server is running?

Send a GET request to the `/health` endpoint (defined in [`api/routers/status.py`](https://github.com/lightningpixel/modly/blob/main/api/routers/status.py)) using `curl http://localhost:PORT/health`. A running server returns `{"status":"ok"}`, which the Electron client uses to verify backend readiness before enabling UI features.