Modly Backend Technologies: A Complete Guide to the Python FastAPI Stack

Modly's backend is a Python-based FastAPI service that runs locally inside the Electron desktop application, providing RESTful endpoints for AI model inference, mesh processing, and extension management through libraries including trimesh, huggingface_hub, and mcp.

The Modly desktop application ships with a self-contained Python backend that powers its 3D generation pipeline. This local server architecture enables GPU-accelerated inference without cloud dependencies while exposing clean HTTP endpoints for the Electron frontend. Understanding the Modly backend technologies reveals how the application orchestrates model downloads, mesh processing, and asynchronous job management entirely on the user's machine.

Core Web Framework and Server Stack

At the heart of Modly's architecture sits FastAPI (version ≥ 0.115.6), configured in [api/main.py](https://github.com/lightningpixel/modly/blob/main/api/main.py) to handle RESTful routing, CORS, and static file serving. The application uses uvicorn as its ASGI server, declared in [api/requirements.txt](https://github.com/lightningpixel/modly/blob/main/api/requirements.txt), which provides the HTTP layer that Electron's renderer process communicates with.

The FastAPI initialization sets up middleware for cross-origin requests and registers routers for generation, model management, and extensions:

from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware

app = FastAPI(title="Modly API", version="0.4.1")
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_methods=["*"],
    allow_headers=["*"],
    expose_headers=["Content-Length"],
)

app.include_router(status.router)
app.include_router(model.router, prefix="/model")
app.include_router(generation.router, prefix="/generate")
app.include_router(extensions.router, prefix="/extensions")

For HTTP client operations, particularly when internal agent scripts call the running API, Modly uses httpx (≥ 0.27.0) rather than synchronous requests libraries.

3D Processing and Geometry Libraries

Modly processes generated meshes using specialized Python geometry libraries. trimesh (≥ 4.5.0) and pymeshlab (≥ 2023.12) handle geometry optimization, decimation, and export operations. These libraries work in concert with the generation pipeline to convert raw model outputs into optimized 3D assets ready for export.

File uploads from the Electron frontend rely on python-multipart to parse multipart/form-data requests containing input images. This integration allows the [api/routers/generation.py](https://github.com/lightningpixel/modly/blob/main/api/routers/generation.py) endpoint to accept binary image data alongside configuration parameters.

Extension System and Process Orchestration

The backend implements a dynamic extension system using the mcp library (≥ 1.0.0) — a lightweight multiprocessing wrapper. The custom ExtensionProcess class in [api/services/extension_process.py](https://github.com/lightningpixel/modly/blob/main/api/services/extension_process.py) manages model extensions either in-process or as isolated subprocesses running in separate virtual environments.

Extension discovery logic in [api/services/generator_registry.py](https://github.com/lightningpixel/modly/blob/main/api/services/generator_registry.py) scans the extensions directory for manifest.json and generator.py files:

def _discover_extensions() -> Dict[str, Tuple[type, dict]]:
    EXTENSIONS_DIR = pathlib.Path.home() / ".modly/extensions"
    for ext_dir in EXTENSIONS_DIR.iterdir():
        manifest_path = ext_dir / "manifest.json"
        generator_path = ext_dir / "generator.py"
        # Load the class (direct mode) or wrap in ExtensionProcess (subprocess mode)

    return result

This architecture allows users to install third-party model generators without restarting the core FastAPI application.

Model Management and Download Infrastructure

To fetch AI model weights from Hugging Face repositories, Modly integrates huggingface_hub (≥ 0.27.0) alongside hf_xet (≥ 0.1.0) for optimized downloads. These libraries handle download state tracking, integrity verification, and resume capabilities for large checkpoint files.

The model management endpoints in the router system coordinate with these libraries to maintain a local cache of weights, typically stored within the user's workspace directory to enable offline inference after initial download.

Asynchronous Job Management

Long-running generation tasks execute as background jobs orchestrated through Python's asyncio and threading modules. The [api/schemas/generation.py](https://github.com/lightningpixel/modly/blob/main/api/schemas/generation.py) file defines the JobStatus Pydantic model that tracks progress, status, and cancellation tokens.

The generation endpoint in [api/routers/generation.py](https://github.com/lightningpixel/modly/blob/main/api/routers/generation.py) creates UUID-based job identifiers and delegates execution to background tasks:

@router.post("/from-image")
async def generate_from_image(
    background_tasks: BackgroundTasks,
    image: UploadFile = File(...),
    model_id: str = Form("sf3d"),
    collection: str = Form("Default"),
    remesh: str = Form("quad"),
    enable_texture: bool = Form(False),
    texture_resolution: int = Form(1024),
):
    job_id = str(uuid.uuid4())
    image_bytes = await image.read()
    background_tasks.add_task(_run_generation, job_id, image_bytes, full_params, collection)
    return {"job_id": job_id}

This pattern enables the API to return immediately with a job identifier while the heavy GPU inference runs asynchronously, supporting status polling and cancellation requests from the frontend.

Security and Networking Stack

Modly implements TLS certificate handling and secure HTTP validation using cryptography (≥ 42.0.0) and certifi (≥ 2024.0.0). These libraries secure local network communications between the Electron renderer and the Python backend, particularly important when handling sensitive path validation and file system operations.

Runtime validation of user-supplied paths ensures all file operations remain confined to the designated workspace directory, preventing path traversal attacks when processing external extensions or user uploads.

File System and Workspace Management

All generated assets and extension data reside in a user-specific workspace located at ~/.modly/workspace. The backend uses pathlib.Path consistently across [api/main.py](https://github.com/lightningpixel/modly/blob/main/api/main.py) and [api/services/generator_registry.py](https://github.com/lightningpixel/modly/blob/main/api/services/generator_registry.py) for cross-platform path handling.

This workspace architecture separates user data from application code, enabling clean uninstallation procedures and allowing users to mount workspaces on external drives for large model caches.

Summary

Modly's backend combines modern Python web frameworks with specialized 3D processing libraries to create a self-contained AI inference server:

• FastAPI (≥ 0.115.6) with uvicorn provides the RESTful HTTP layer and CORS handling in api/main.py
• trimesh and pymeshlab handle geometry processing and mesh optimization
• mcp (≥ 1.0.0) and the ExtensionProcess class enable isolated extension execution
• huggingface_hub and hf_xet manage model weight downloads and integrity verification
• asyncio with custom JobStatus schemas power the asynchronous generation pipeline
• cryptography and certifi secure local network communications
• python-multipart and pathlib handle file uploads and workspace organization

Together, these technologies create a pure-Python, GPU-accelerated inference pipeline that operates entirely on local hardware while exposing a standard HTTP API for desktop and command-line interfaces.

Frequently Asked Questions

What Python version does Modly require?

Modly targets modern Python versions compatible with FastAPI 0.115.6 and the async/await patterns used throughout the codebase. The specific Python version constraints should be checked in the repository's pyproject.toml or setup documentation, though the codebase utilizes features from Python 3.8+ given the type hint syntax and pathlib usage.

Can the Modly backend run independently of the Electron app?

Yes, the FastAPI backend in api/main.py can operate as a standalone server. Because it exposes standard HTTP endpoints, external scripts and the modly-cli tool can communicate with it directly. The Electron app simply manages the lifecycle of the uvicorn process and provides the user interface layer.

How does Modly handle large file uploads for 3D generation?

Modly uses python-multipart to stream image uploads through the /generate/from-image endpoint. The backend reads file contents asynchronously (await image.read()) before passing bytes to the generation pipeline. For large mesh exports, the system utilizes the workspace directory at ~/.modly/workspace rather than holding complete files in memory.

What makes Modly's extension system different from standard Python plugins?

Unlike typical import-based plugins, Modly's ExtensionProcess class (defined in api/services/extension_process.py) can launch extensions in isolated subprocesses with separate virtual environments. This prevents dependency conflicts between models requiring different PyTorch or CUDA versions while still exposing them through the unified FastAPI router interface.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}