Lightningpixel/Modly Project Structure: Electron, React, and FastAPI Architecture Explained

The Modly repository is organized as a dual-runtime desktop application with a React/Electron frontend in src/ and electron/, a Python FastAPI backend in api/, and CLI automation tools in tools/, communicating via IPC and HTTP to convert images into 3D meshes.

The lightningpixel/modly repository implements a modular desktop application that separates UI rendering from heavy AI computation. The codebase follows a clean architecture pattern that isolates the React/TypeScript frontend, Electron main process, and Python FastAPI backend into distinct top-level directories. This structure enables both interactive use through the desktop GUI and headless automation via the command-line interface.

High-Level Architecture Overview

Modly operates as a dual-runtime system where the Node.js/Electron process hosts the user interface while spawning a local Python server as a child process for AI inference. The frontend communicates with the backend through Electron IPC channels and HTTP endpoints exposed by FastAPI.

The repository is divided into these primary domains:

• src/ – React/TypeScript UI components, workflow engine, and state management
• electron/ – Main process code for window creation, IPC handlers, and native utilities
• api/ – Python FastAPI server handling model loading, generation, and extensions
• tools/ – CLI utilities for headless interaction with the running application
• scripts/ – Build automation, packaging helpers, and environment setup

Frontend: React, Three.js, and Workflow Engine

The user interface resides entirely within src/ and leverages React, Three.js, and Zustand for state management. The directory organizes functionality by domain rather than technical role.

Core Directory Layout

• src/areas/ – Domain-specific modules:
  • workflows/ – Node-based workflow editor with processors like mesh-optimizer and mesh-exporter located in src/areas/workflows/nodes/
  • models/ – Model selection and download UI
  • generate/ – Service hooks (e.g., useGeneration) that convert UI actions into API calls
• src/shared/ – Reusable Zustand stores (appStore.ts, navStore.ts, extensionsStore.ts), hooks, and UI components

State management uses Zustand for predictable updates across the component tree:

// src/areas/workflows/workflowRunStore.ts
import { create } from 'zustand';
export const useWorkflowRunStore = create(() => ({
  runs: [],
  addRun: (run) => set(state => ({ runs: [...state.runs, run] })),
}));

3D mesh visualization relies on Three.js and React Three Fiber (@react-three/fiber, @react-three/drei), rendering generated geometries within the Electron window.

Electron Main Process

The electron/main/ directory contains the Node.js/Electron bootstrap code that creates the application window and manages native system integration.

Key modules include:

• index.ts – Entry point that initializes the BrowserWindow and loads the React UI
• ipc-handlers.ts – Registers IPC channels for UI-to-main communication, such as triggering workflow runs
• artifact-registry-service.ts – Manages the lifecycle and storage of generated mesh files
• extension-path-guard.ts – Security utility that validates external extensions remain within a sandboxed path, preventing arbitrary code execution

The main process spawns the Python backend as a subprocess and mediates all filesystem access and native APIs on behalf of the renderer.

Python Backend: FastAPI Services

Located under api/, the Python layer exposes REST endpoints for AI model management, image-to-mesh generation, and extension execution.

API Structure

• api/main.py – FastAPI application entry point that configures CORS and mounts routers
• api/routers/ – HTTP endpoint definitions:
  • model.py – Model listing and download management
  • generation.py – Image upload and mesh generation endpoints (e.g., POST /workflow-runs/from-image)
  • extension.py – Extension installation and execution APIs
• api/services/ – Business logic including generator_registry.py for model loading and inference orchestration

The Electron UI communicates with these endpoints via HTTP requests, isolating heavy PyTorch/AI workloads from the JavaScript main thread.

CLI and Automation Tools

The tools/modly-cli/ directory provides a headless interface for driving Modly without the graphical UI. The CLI interacts with the running application through the same HTTP API used by the frontend.

The primary entry point is tools/modly-cli/agent.py, which implements commands for health checks, model enumeration, workflow status monitoring, and direct generation:

python tools/modly-cli/agent.py generate --image ./input.png --output ./result.glb

This agent contract allows CI/CD pipelines and automated workflows to leverage Modly's AI capabilities programmatically.

Build System and Configuration

Packaging and development workflows are coordinated through standard toolchain configuration files:

• package.json – npm manifest defining React/Three.js dependencies and scripts (dev, build, test, package)
• tsconfig*.json – TypeScript compiler configurations for the main app, built-in extensions, and Node environment
• scripts/ – Build utilities including build-builtins.mjs for bundling and Python embed download scripts

Development requires coordinating both runtimes:

# Install frontend dependencies

npm install

# Set up Python backend

cd api
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate

pip install -r requirements.txt

# Launch with hot-reload

npm run dev

Extending Modly with Custom Workflow Nodes

Developers can add functionality by creating new workflow nodes in TypeScript. Each node implements the NodeProcessor interface and resides under src/areas/workflows/nodes/:

// src/areas/workflows/nodes/custom-node/processor.ts
import { NodeProcessor } from '../../nodeBehaviors';

export const customProcessor: NodeProcessor = async (inputs) => {
  // Access inputs.mesh and perform processing
  const result = await myHeavyComputation(inputs.mesh);
  return { mesh: result };
};

These processors execute within the Electron renderer process and can communicate with the Python backend via the shared API client.

Summary

• Modly (lightningpixel/modly) uses a dual-runtime architecture combining Electron/React with Python FastAPI
• The src/ directory contains the React frontend with Zustand state stores and Three.js rendering
• electron/main/ handles window management, IPC communication, and security sandboxing
• api/ provides FastAPI endpoints for AI inference, model management, and extensions
• tools/modly-cli/ offers headless automation via Python scripts that interact with the running API
• Communication flows through Electron IPC (UI to main) and HTTP (main to Python backend)

Frequently Asked Questions

How do the Electron frontend and Python backend communicate?

The Electron main process spawns the Python FastAPI server as a child process. The React frontend sends messages to the main process via Electron IPC channels defined in electron/main/ipc-handlers.ts, while the main process forwards generation requests to the Python backend via HTTP requests to localhost endpoints (e.g., POST /workflow-runs/from-image).

What is the purpose of the tools/modly-cli/agent.py script?

The agent.py CLI tool provides a command-line interface for headless operation. It implements commands like health, model list, and generate that interact with the running Modly instance through the same REST API used by the GUI, enabling automation and scripting without launching the desktop interface.

Where should new workflow nodes be added in the Modly project structure?

New workflow nodes belong under src/areas/workflows/nodes/ with a subdirectory containing a processor.ts file. Each processor must implement the NodeProcessor interface from src/areas/workflows/nodeBehaviors, accepting inputs and returning processed mesh data that flows through the workflow graph.

What dependencies are required to run Modly in development?

Development requires Node.js (for npm install and the React build), Python 3 with a virtual environment (for FastAPI and PyTorch dependencies listed in api/requirements.txt), and standard C++ build tools for native npm modules. The npm run dev script handles concurrent startup of both the Vite dev server and the Python API.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}