# Lightningpixel/Modly Project Structure: Electron, React, and FastAPI Architecture Explained

> Explore the lightningpixel/modly project structure, a dual-runtime desktop app. Discover its Electron React frontend, Python FastAPI backend, and CLI tools architecture for image to 3D mesh conversion.

- Repository: [lightningpixel/modly](https://github.com/lightningpixel/modly)
- Tags: architecture
- Published: 2026-08-19

---

**The Modly repository is organized as a dual-runtime desktop application with a React/Electron frontend in `src/` and `electron/`, a Python FastAPI backend in `api/`, and CLI automation tools in `tools/`, communicating via IPC and HTTP to convert images into 3D meshes.**

The **lightningpixel/modly** repository implements a modular desktop application that separates UI rendering from heavy AI computation. The codebase follows a clean architecture pattern that isolates the **React/TypeScript frontend**, **Electron main process**, and **Python FastAPI backend** into distinct top-level directories. This structure enables both interactive use through the desktop GUI and headless automation via the command-line interface.

## High-Level Architecture Overview

Modly operates as a **dual-runtime system** where the Node.js/Electron process hosts the user interface while spawning a local Python server as a child process for AI inference. The frontend communicates with the backend through **Electron IPC** channels and **HTTP endpoints** exposed by FastAPI.

The repository is divided into these primary domains:

- **`src/`** – React/TypeScript UI components, workflow engine, and state management
- **`electron/`** – Main process code for window creation, IPC handlers, and native utilities
- **`api/`** – Python FastAPI server handling model loading, generation, and extensions
- **`tools/`** – CLI utilities for headless interaction with the running application
- **`scripts/`** – Build automation, packaging helpers, and environment setup

## Frontend: React, Three.js, and Workflow Engine

The user interface resides entirely within **`src/`** and leverages **React**, **Three.js**, and **Zustand** for state management. The directory organizes functionality by domain rather than technical role.

### Core Directory Layout

- **`src/areas/`** – Domain-specific modules:
  - **`workflows/`** – Node-based workflow editor with processors like `mesh-optimizer` and `mesh-exporter` located in `src/areas/workflows/nodes/`
  - **`models/`** – Model selection and download UI
  - **`generate/`** – Service hooks (e.g., `useGeneration`) that convert UI actions into API calls
- **`src/shared/`** – Reusable Zustand stores ([`appStore.ts`](https://github.com/lightningpixel/modly/blob/main/appStore.ts), [`navStore.ts`](https://github.com/lightningpixel/modly/blob/main/navStore.ts), [`extensionsStore.ts`](https://github.com/lightningpixel/modly/blob/main/extensionsStore.ts)), hooks, and UI components

State management uses **Zustand** for predictable updates across the component tree:

```typescript
// src/areas/workflows/workflowRunStore.ts
import { create } from 'zustand';
export const useWorkflowRunStore = create(() => ({
  runs: [],
  addRun: (run) => set(state => ({ runs: [...state.runs, run] })),
}));

```

3D mesh visualization relies on **Three.js** and **React Three Fiber** (`@react-three/fiber`, `@react-three/drei`), rendering generated geometries within the Electron window.

## Electron Main Process

The **`electron/main/`** directory contains the Node.js/Electron bootstrap code that creates the application window and manages native system integration.

Key modules include:

- **[`index.ts`](https://github.com/lightningpixel/modly/blob/main/index.ts)** – Entry point that initializes the `BrowserWindow` and loads the React UI
- **[`ipc-handlers.ts`](https://github.com/lightningpixel/modly/blob/main/ipc-handlers.ts)** – Registers IPC channels for UI-to-main communication, such as triggering workflow runs
- **[`artifact-registry-service.ts`](https://github.com/lightningpixel/modly/blob/main/artifact-registry-service.ts)** – Manages the lifecycle and storage of generated mesh files
- **[`extension-path-guard.ts`](https://github.com/lightningpixel/modly/blob/main/extension-path-guard.ts)** – Security utility that validates external extensions remain within a sandboxed path, preventing arbitrary code execution

The main process spawns the Python backend as a subprocess and mediates all filesystem access and native APIs on behalf of the renderer.

## Python Backend: FastAPI Services

Located under **`api/`**, the Python layer exposes **REST endpoints** for AI model management, image-to-mesh generation, and extension execution.

### API Structure

- **[`api/main.py`](https://github.com/lightningpixel/modly/blob/main/api/main.py)** – FastAPI application entry point that configures CORS and mounts routers
- **`api/routers/`** – HTTP endpoint definitions:
  - **[`model.py`](https://github.com/lightningpixel/modly/blob/main/model.py)** – Model listing and download management
  - **[`generation.py`](https://github.com/lightningpixel/modly/blob/main/generation.py)** – Image upload and mesh generation endpoints (e.g., `POST /workflow-runs/from-image`)
  - **[`extension.py`](https://github.com/lightningpixel/modly/blob/main/extension.py)** – Extension installation and execution APIs
- **`api/services/`** – Business logic including [`generator_registry.py`](https://github.com/lightningpixel/modly/blob/main/generator_registry.py) for model loading and inference orchestration

The Electron UI communicates with these endpoints via HTTP requests, isolating heavy PyTorch/AI workloads from the JavaScript main thread.

## CLI and Automation Tools

The **`tools/modly-cli/`** directory provides a **headless interface** for driving Modly without the graphical UI. The CLI interacts with the running application through the same HTTP API used by the frontend.

The primary entry point is **[`tools/modly-cli/agent.py`](https://github.com/lightningpixel/modly/blob/main/tools/modly-cli/agent.py)**, which implements commands for health checks, model enumeration, workflow status monitoring, and direct generation:

```bash
python tools/modly-cli/agent.py generate --image ./input.png --output ./result.glb

```

This agent contract allows CI/CD pipelines and automated workflows to leverage Modly's AI capabilities programmatically.

## Build System and Configuration

Packaging and development workflows are coordinated through standard toolchain configuration files:

- **[`package.json`](https://github.com/lightningpixel/modly/blob/main/package.json)** – npm manifest defining React/Three.js dependencies and scripts (`dev`, `build`, `test`, `package`)
- **`tsconfig*.json`** – TypeScript compiler configurations for the main app, built-in extensions, and Node environment
- **`scripts/`** – Build utilities including `build-builtins.mjs` for bundling and Python embed download scripts

Development requires coordinating both runtimes:

```bash

# Install frontend dependencies

npm install

# Set up Python backend

cd api
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate

pip install -r requirements.txt

# Launch with hot-reload

npm run dev

```

## Extending Modly with Custom Workflow Nodes

Developers can add functionality by creating new workflow nodes in TypeScript. Each node implements the `NodeProcessor` interface and resides under `src/areas/workflows/nodes/`:

```typescript
// src/areas/workflows/nodes/custom-node/processor.ts
import { NodeProcessor } from '../../nodeBehaviors';

export const customProcessor: NodeProcessor = async (inputs) => {
  // Access inputs.mesh and perform processing
  const result = await myHeavyComputation(inputs.mesh);
  return { mesh: result };
};

```

These processors execute within the Electron renderer process and can communicate with the Python backend via the shared API client.

## Summary

- **Modly** (`lightningpixel/modly`) uses a **dual-runtime architecture** combining Electron/React with Python FastAPI
- The **`src/`** directory contains the React frontend with **Zustand** state stores and **Three.js** rendering
- **`electron/main/`** handles window management, IPC communication, and security sandboxing
- **`api/`** provides FastAPI endpoints for AI inference, model management, and extensions
- **`tools/modly-cli/`** offers headless automation via Python scripts that interact with the running API
- Communication flows through **Electron IPC** (UI to main) and **HTTP** (main to Python backend)

## Frequently Asked Questions

### How do the Electron frontend and Python backend communicate?

The Electron main process spawns the Python FastAPI server as a child process. The React frontend sends messages to the main process via **Electron IPC** channels defined in [`electron/main/ipc-handlers.ts`](https://github.com/lightningpixel/modly/blob/main/electron/main/ipc-handlers.ts), while the main process forwards generation requests to the Python backend via **HTTP requests** to localhost endpoints (e.g., `POST /workflow-runs/from-image`).

### What is the purpose of the [`tools/modly-cli/agent.py`](https://github.com/lightningpixel/modly/blob/main/tools/modly-cli/agent.py) script?

The **[`agent.py`](https://github.com/lightningpixel/modly/blob/main/agent.py)** CLI tool provides a command-line interface for headless operation. It implements commands like `health`, `model list`, and `generate` that interact with the running Modly instance through the same REST API used by the GUI, enabling automation and scripting without launching the desktop interface.

### Where should new workflow nodes be added in the Modly project structure?

New workflow nodes belong under **`src/areas/workflows/nodes/`** with a subdirectory containing a [`processor.ts`](https://github.com/lightningpixel/modly/blob/main/processor.ts) file. Each processor must implement the `NodeProcessor` interface from `src/areas/workflows/nodeBehaviors`, accepting inputs and returning processed mesh data that flows through the workflow graph.

### What dependencies are required to run Modly in development?

Development requires **Node.js** (for `npm install` and the React build), **Python 3** with a virtual environment (for FastAPI and PyTorch dependencies listed in [`api/requirements.txt`](https://github.com/lightningpixel/modly/blob/main/api/requirements.txt)), and standard C++ build tools for native npm modules. The `npm run dev` script handles concurrent startup of both the Vite dev server and the Python API.