Modly API and Backend Architecture: Key Files and How They Work

The Modly backend combines an Electron main process with a FastAPI Python server, where key files like electron/main/python-bridge.ts, api/main.py, and src/shared/hooks/useApi.ts orchestrate generation, model management, and IPC communication.

Modly is a hybrid desktop application that bridges JavaScript and Python runtimes to power AI-driven 3D generation. According to the lightningpixel/modly source code, the architecture layers a React/Vue renderer, an Electron main process, and a uvicorn-powered FastAPI server to handle mesh generation, model downloads, and extension lifecycle events.

Architecture Overview

The Modly API backend operates as a multi-process hybrid. The Electron main process (Node.js) manages system-level operations—filesystem access, native dialogs, and extension installation—while a Python FastAPI server handles compute-intensive tasks like image-to-mesh generation and model inference.

The communication flow follows this path:

1. Renderer calls methods via useApi() (TypeScript hook).
2. Axios sends HTTP requests to 127.0.0.1:8765 (the FastAPI server).
3. Python Bridge (electron/main/python-bridge.ts) spawns and monitors the uvicorn process.
4. FastAPI Routers (api/routers/*.py) dispatch requests to the appropriate Service layer (api/services/*.py).
5. IPC Channels (electron/main/ipc-handlers.ts) bridge non-HTTP operations like GitHub extension installs.

Core Backend Components

Frontend API Integration: src/shared/hooks/useApi.ts

The renderer consumes the Modly API backend through a centralized hook. This file wraps axios to build multipart/form-data requests for image uploads and provides typed methods including generateFromImage(), pollJobStatus(), downloadModel(), and optimizeMesh().

When a user drops an image, the flow begins here:

// src/shared/hooks/useApi.ts
const { jobId } = await client.post<{ job_id: string }>(
  '/generate/from-image',
  formData,
  { headers: { 'Content-Type': 'multipart/form-data' } }
);

Electron-to-Python Bridge: electron/main/python-bridge.ts

This module is responsible for bootstrapping the FastAPI server. It spawns a Python process running uvicorn, injects environment variables for model paths (MODELS_DIR), workspace directories, Hugging Face tokens, and extension folders, then polls the /health endpoint until the server is ready.

Key constants include PythonBridge.API_BASE_URL (set to http://127.0.0.1:8765) and the start() method that initiates the subprocess.

IPC Handlers: electron/main/ipc-handlers.ts

System-level operations that cannot use HTTP are exposed via Electron IPC channels. This file registers handlers for model:download, fs:selectImage, and extensions:installFromGitHub.

The extension installation pipeline validates manifests, stages files in a temporary directory, performs atomic moves into extensionsDir, and runs optional setup.py or npm installs before cleanup.

FastAPI Application Entry: api/main.py

This file creates the FastAPI application instance, attaches routers from api/routers/, and configures CORS to allow cross-origin requests from the Electron renderer. It serves as the central dispatch point for all REST traffic.

Generation Router: api/routers/generation.py

The core generation endpoints live here. It implements:

• POST /generate/from-image – Accepts multipart uploads and initiates mesh generation jobs.
• GET /generate/status/<jobId> – Streams progress updates to the frontend.
• Cancellation logic for active jobs.

Requests are delegated to the Generator Registry to resolve the correct pipeline based on the provided modelId.

Model Management: api/routers/model.py

This router exposes endpoints for checking download status, listing available models, downloading weights from Hugging Face, and unloading models from VRAM. The renderer calls these via useApi().getModelStatus().

Mesh Optimization: api/routers/optimize.py

Post-processing operations reside in this module, handling routes like /optimize/mesh, /optimize/smooth, and /optimize/transform to refine generated geometry before export.

Extension System Routers: api/routers/extensions.py

Extensions integrate with the Modly API backend through this router, which exposes metadata endpoints, installation triggers, and reload commands for hot-reloading Python-based model extensions.

Service Layer: Generator Registry

api/services/generator_registry.py

This service maintains an in-memory registry of available generators (image-to-mesh, text-to-mesh, etc.). When a request arrives, it resolves the appropriate generator class based on the model ID and executes the pipeline.

Service Layer: Extension Process Manager

api/services/extension_process.py

For process-type extensions, this service manages child Python subprocesses. It handles log forwarding to the main process, monitors health, and enforces safe termination when extensions are disabled or uninstalled.

Code Examples in Context

Initiating Generation from the Renderer

// Component using the Modly API backend
const api = useApi();
const onDrop = async (filePath: string) => {
  const job = await api.generateFromImage(filePath, {
    modelId: 'stable-diffusion',
    remesh: true,
    enableTexture: true,
    textureResolution: 1024,
  });
  const status = await api.pollJobStatus(job.jobId);
};

Installing Extensions via IPC

Renderer side:

window.electron.ipcRenderer.invoke('extensions:installFromGitHub', repoUrl)
  .then((result) => {
    if (result.success) console.log('Installed:', result.extensionId);
  });

Main process handler:

// electron/main/ipc-handlers.ts
ipcMain.handle('extensions:installFromGitHub', async (event, githubUrl) => {
  // 1. Validate URL, 2. Download tarball, 3. Extract, 4. Validate manifest
  // 5. Stage temp dir, 6. Atomic rename, 7. Run setup.py/npm install, 8. Cleanup
});

Health Check Polling

// electron/main/python-bridge.ts
await axios.get(`${API_BASE_URL}/health`, { timeout: 2000 });

Complete File Reference

The following files constitute the core Modly API backend:

• src/shared/hooks/useApi.ts – Frontend axios wrapper for HTTP requests to FastAPI.
• electron/main/python-bridge.ts – Spawns the Python uvicorn server and manages lifecycle.
• electron/main/ipc-handlers.ts – Registers IPC channels for filesystem and extension operations.
• api/main.py – FastAPI application factory with router registration.
• api/routers/generation.py – Image-to-mesh generation and job status endpoints.
• api/routers/model.py – Model listing, download, and unload operations.
• api/routers/optimize.py – Mesh smoothing and transformation endpoints.
• api/routers/extensions.py – Extension metadata and reload triggers.
• api/services/generator_registry.py – Resolution and execution of generation pipelines.
• api/services/extension_process.py – Subprocess management for extensions.

Summary

• Modly uses a hybrid architecture: Electron Node.js for system operations, FastAPI Python for ML workloads.
• Key entry points include electron/main/python-bridge.ts (server lifecycle), src/shared/hooks/useApi.ts (frontend client), and api/main.py (REST application).
• Generation flow moves from the renderer through useApi() → FastAPI routers (api/routers/generation.py) → generator_registry.py services.
• Extensions install via atomic file operations in ipc-handlers.ts and run as managed child processes via extension_process.py.

Frequently Asked Questions

How does the Modly frontend communicate with the Python backend?

The renderer uses the useApi() hook in src/shared/hooks/useApi.ts to send HTTP requests to a local FastAPI server running on 127.0.0.1:8765. The Electron main process spawns this Python server via electron/main/python-bridge.ts and manages its lifecycle, while the frontend treats it as a standard REST API.

What file is responsible for launching the FastAPI server in Modly?

The electron/main/python-bridge.ts module spawns the uvicorn process. It sets required environment variables—including paths to models, workspace directories, and Hugging Face tokens—and polls the /health endpoint to confirm the server is ready before allowing renderer connections.

Where is the generation logic implemented in Modly's backend?

Route definitions exist in api/routers/generation.py, which handles HTTP requests for /generate/from-image and status polling. The actual pipeline execution is delegated to api/services/generator_registry.py, which resolves the correct generator class based on the requested model ID and manages the inference workflow.

How are extensions installed and managed in Modly?

Extension installation flows through electron/main/ipc-handlers.ts, which validates GitHub URLs, downloads tarballs, validates manifests, and performs atomic directory swaps. For runtime management, api/services/extension_process.py spawns child Python processes for each extension, forwarding logs and enforcing safe termination when extensions are disabled.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}