Modly API Structure and Backend Architecture Explained

Modly's backend is a FastAPI server embedded within the Electron client, centered on a GeneratorRegistry that dynamically discovers and manages AI model extensions through a modular REST API organized under the api/routers/ package.

The lightningpixel/modly repository implements a lightweight yet extensible backend architecture that powers its AI-driven 3D generation capabilities. At its core lies a FastAPI application defined in api/main.py that exposes REST endpoints for model management, generation tasks, and system health, all while running locally inside the Electron desktop application. This design decouples the AI generation logic into swappable extensions while providing a standardized HTTP interface for the frontend to consume.

Core Backend Components

FastAPI Application Entry Point (api/main.py)

The backend lifecycle begins in api/main.py, which instantiates the central FastAPI application, configures CORS middleware for cross-origin requests, and mounts the workspace directory for serving generated assets. This file aggregates all modular routers from the api/routers/ package and defines the server's startup behavior, creating a unified HTTP interface on a local port that the Electron renderer communicates with via the preload script.

Router Organization (api/routers/)

Functional endpoints are grouped by domain across separate router modules under api/routers/. Each router handles a specific concern: generation.py manages job submission and status polling, model.py handles model switching and downloads, status.py provides health checks, and additional routers like agent.py support workflow automation. This separation of concerns allows the FastAPI layer to focus strictly on HTTP validation and request routing while delegating stateful operations to underlying services.

GeneratorRegistry Service (api/services/generator_registry.py)

The architectural heart of the system resides in api/services/generator_registry.py, which implements the GeneratorRegistry class. This service discovers extension modules within the EXTENSIONS_DIR, dynamically loads them either directly in-process or via subprocess isolation, and maintains the active model state. It tracks downloaded models, manages the BaseGenerator interface for all AI adapters, and provides thread-safe methods like switch_model() and get_active() that routers invoke to execute generation tasks.

Request Flow Architecture

When a user submits a generation request, the flow traverses three distinct layers. First, the client POSTs to /generate/from-image defined in api/routers/generation.py, which validates the payload and calls generator_registry.switch_model() to activate the requested AI extension.

Next, the endpoint schedules _run_generation() as a background task, which retrieves the active generator via generator_registry.get_active(), ensures the model weights are downloaded to ~/.modly/models, and invokes the generator's generate() method. Finally, the generator writes outputs to a subdirectory of WORKSPACE_DIR (defaulting to ~/.modly/workspace), and the router updates an in-memory _jobs dictionary with status and output URLs accessible via /generate/status/{job_id}.

Key REST Endpoints and Usage Examples

The Modly API exposes standard HTTP endpoints for health monitoring, model management, and asynchronous generation.

Check System Health

curl -s http://localhost:PORT/health

# → {"status":"ok"}

List Available Models

curl -s http://localhost:PORT/model/all

# → [{"id":"sf3d","name":"ShapeFrom3D", "downloaded":true, "loaded":true, ...}, ...]

Switch Active Model

curl -X POST -H "Content-Type: application/json" \
     -d '{"model_id":"my‑custom‑model"}' \
     http://localhost:PORT/model/switch

# → {"active":"my‑custom‑model"}

Submit Image-to-Mesh Generation Job

curl -X POST -F image=@input.png \
     -F model_id=sf3d \
     -F collection=MyProject \
     -F remesh=quad \
     -F enable_texture=true \
     -F texture_resolution=1024 \
     http://localhost:PORT/generate/from-image

# → {"job_id":"c8f2a9d4‑…"}

Poll Job Status

curl http://localhost:PORT/generate/status/<job_id>

# Example response:

# {

#   "job_id":"c8f2a9d4‑…",

#   "status":"done",

#   "progress":100,

#   "output_url":"/workspace/MyProject/mesh.obj"

# }

Stream Model Downloads (SSE)

curl -N http://localhost:PORT/model/hf-download?repo_id=owner/repo&model_id=sf3d

# Receives lines like: data: {"percent":45,"file":"model.bin","status":"Downloading…"}

Configuration and Storage

The backend configures two primary filesystem locations defined in api/services/generator_registry.py (lines 24-30). The WORKSPACE_DIR (default ~/.modly/workspace) stores generated assets and project outputs, while separate model weights reside in ~/.modly/models. These paths are configurable and exposed through the FastAPI static file middleware mounted in api/main.py, allowing the frontend to retrieve generated meshes and textures via simple HTTP GET requests to /workspace/<path>.

Summary

• Modly's backend utilizes a FastAPI server running locally within the Electron client, initialized in api/main.py with CORS and router registration.
• Router modules under api/routers/ organize endpoints by function (generation, model, status), keeping HTTP concerns separated from business logic.
• The GeneratorRegistry in api/services/generator_registry.py dynamically discovers, loads, and manages AI model extensions through a consistent BaseGenerator interface.
• Asynchronous generation uses FastAPI background tasks with in-memory job tracking in _jobs, writing outputs to configurable workspace directories.
• The architecture supports dynamic extension loading, allowing new AI models to be added by placing them in the EXTENSIONS_DIR without modifying core API code.

Frequently Asked Questions

What framework powers the Modly backend?

The Modly backend is built on FastAPI, a modern Python web framework. The application initializes in api/main.py with CORS middleware and router registration, providing a high-performance asynchronous server that handles REST requests from the Electron frontend through a local HTTP port.

How does Modly manage different AI models dynamically?

Modly uses the GeneratorRegistry class (api/services/generator_registry.py) to scan the EXTENSIONS_DIR for available extensions, instantiate their BaseGenerator implementations, and toggle between them via the switch_model() method. This registry supports both in-process and subprocess loading modes for isolation and resource management.

Where does Modly store generated files and model weights?

By default, Modly stores generated assets in ~/.modly/workspace and downloaded model weights in ~/.modly/models, as defined in the generator registry configuration. The FastAPI app mounts the workspace directory as a static file route in api/main.py, enabling direct URL access to outputs like /workspace/MyProject/mesh.obj.

How do I check if the Modly API server is running?

Send a GET request to the /health endpoint (defined in api/routers/status.py) using curl http://localhost:PORT/health. A running server returns {"status":"ok"}, which the Electron client uses to verify backend readiness before enabling UI features.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}