# How the Modly CLI Agent Communicates with a Running Desktop App: HTTP API Architecture Explained

> Discover how the Modly CLI agent communicates with your desktop app. Explore the HTTP API architecture using FastAPI and REST for seamless integration.

- Repository: [lightningpixel/modly](https://github.com/lightningpixel/modly)
- Tags: architecture
- Published: 2026-08-20

---

**The Modly CLI agent communicates with the running desktop app by sending HTTP requests to a FastAPI server embedded in the Electron-based desktop client, using a REST-style API at `http://127.0.0.1:8765` by default.**

This design allows the CLI to control the same backend that powers the desktop GUI, enabling automation and scripting without requiring the UI to be open. Understanding this communication architecture is essential for building custom workflows, debugging connectivity issues, or extending Modly's capabilities.

---

## Key Communication Architecture

The Modly desktop application is built with **Electron** and embeds a **FastAPI** server that exposes REST endpoints such as `/health`, `/model/status`, and `/generate/...`. The CLI agent in [`tools/modly-cli/agent.py`](https://github.com/lightningpixel/modly/blob/main/tools/modly-cli/agent.py) interacts with this server through standard HTTP requests—not through Electron's IPC channels.

This separation provides two major benefits:

- **Language-agnostic integration** — Any tool that can make HTTP requests can control Modly
- **UI independence** — The backend can run in "headless" mode without the Electron frontend

---

## How the CLI Agent Establishes Connection

The communication flow follows a consistent pattern across all CLI operations:

### 1. Server Discovery and Startup

If the FastAPI server is not running, the CLI can launch it automatically. The `cmd_serve` function (lines 409–420 in [`agent.py`](https://github.com/lightningpixel/modly/blob/main/agent.py)) resolves configuration via `_resolve_serve_config` and starts the backend with `_start_backend`:

```python

# From tools/modly-cli/agent.py

def cmd_serve(args):
    config = _resolve_serve_config(args)
    if args.detach:
        proc = _start_backend(config)  # Spawns uvicorn process

        return {"ok": True, "pid": proc.pid, "base_url": config["base_url"]}

```

This spawns a Python process running `uvicorn main:app`, binding to `127.0.0.1:8765` by default.

### 2. Health Verification

Before executing commands, the CLI validates server availability through `_try_health` and `_require_health`:

```bash
$ modly-cli health
{
  "ok": true,
  "base_url": "http://127.0.0.1:8765",
  "health": { "status": "ready", "version": "1.2.3" }
}

```

### 3. API Request Execution

All functional commands use the `_request_json` helper (lines 66–84 in [`agent.py`](https://github.com/lightningpixel/modly/blob/main/agent.py)), which wraps Python's `urllib.request`:

```python
def _request_json(method, path, body=None, headers=None, timeout=DEFAULT_TIMEOUT_SECONDS):
    url = f"{BASE_URL}{path}"
    req = urllib.request.Request(url, method=method, data=body, headers=headers)
    with urllib.request.urlopen(req, timeout=timeout) as response:
        return json.loads(response.read())

```

This pattern supports **GET**, **POST**, **PUT**, and **DELETE** operations against FastAPI endpoints defined in `api/routers/*.py`.

---

## Handling File Uploads and Streaming Data

For commands requiring file inputs, the CLI constructs **multipart/form-data** requests using `_multipart_form` (lines 115–123 in [`agent.py`](https://github.com/lightningpixel/modly/blob/main/agent.py)):

```python
def _multipart_form(fields, files):
    # fields: dict of form field name → value

    # files: dict of field name → (filename, file_bytes, content_type)

    body, content_type = encode_multipart_formdata(fields, files)
    return body, {"Content-Type": content_type}

```

These uploads target endpoints like:

- `POST /generate/from-image` — Image-to-mesh generation
- `POST /workflow-runs/from-image` — Workflow execution with image input

Example CLI invocation:

```bash
$ modly-cli generate --image ./portrait.jpg --output ./portrait.glb
{
  "ok": true,
  "base_url": "http://127.0.0.1:8765",
  "run": { "kind": "workflowRun", "id": "12345" },
  "status": { "status": "done" },
  "workspace_path": "workspace/Agent/portrait.glb",
  "meta": { "status_command": "modly-cli workflow-run status 12345" }
}

```

---

## Polling and Job Progress Monitoring

Long-running operations use a polling loop to track completion. The `_poll_workflow_run` function (lines 735–751 in [`agent.py`](https://github.com/lightningpixel/modly/blob/main/agent.py)) repeatedly queries status endpoints:

```python
def _poll_workflow_run(run_id, poll_interval=1.0, timeout=None):
    start = time.time()
    while True:
        status = _request_json("GET", f"/workflow-runs/{run_id}")
        if status["status"] in ("done", "failed", "cancelled"):
            return status
        if timeout and (time.time() - start) > timeout:
            raise TimeoutError(f"Polling exceeded {timeout}s")
        time.sleep(poll_interval)
        # Progress optionally emitted to stderr

```

Equivalent manual status check:

```bash
$ modly-cli generate status --job-id 7f9c3a
{
  "ok": true,
  "job_id": "7f9c3a",
  "status": { "status": "running", "progress": 45, "stage": "mesh_reconstruction" }
}

```

---

## Recovery Metadata and Reproducibility

Every response includes a `meta` object generated by `_recovery_meta`. This contains exact CLI commands to reproduce or inspect the operation:

```json
{
  "meta": {
    "status_command": "modly-cli workflow-run status 12345",
    "cancel_command": "modly-cli workflow-run cancel 12345",
    "logs_command": "modly-cli workflow-run logs 12345"
  }
}

```

This self-documenting design enables script automation and debugging without manual URL construction.

---

## What the CLI Does NOT Use: Electron IPC

The CLI intentionally bypasses Electron's IPC system. The desktop UI uses separate channels defined in:

- [`electron/main/ipc-handlers.ts`](https://github.com/lightningpixel/modly/blob/main/electron/main/ipc-handlers.ts) — Main-process IPC handlers
- [`electron/preload/electron-api.ts`](https://github.com/lightningpixel/modly/blob/main/electron/preload/electron-api.ts) — Renderer-to-main bridge

These are **UI-only** and irrelevant to CLI communication. The FastAPI server serves both the Electron frontend and external HTTP clients uniformly.

---

## Running the Backend Without the Desktop UI

For server environments or automation pipelines, start the backend directly:

```bash
$ modly-cli serve --detach --print-command
{
  "ok": true,
  "cmd": ["/path/to/python", "-m", "uvicorn", "main:app", "--host", "127.0.0.1", "--port", "8765"],
  "cwd": "/path/to/modly/api",
  "env": { "MODELS_DIR": "/models", "WORKSPACE_DIR": "/workspace" },
  "base_url": "http://127.0.0.1:8765"
}

```

This produces a headless Modly instance controllable entirely via HTTP.

---

## Summary

- The **Modly CLI agent** communicates through standard **HTTP requests** to a **FastAPI server** running inside the Electron desktop app
- Default base URL is **`http://127.0.0.1:8765`**, configurable via environment or CLI flags
- Core communication code lives in **[`tools/modly-cli/agent.py`](https://github.com/lightningpixel/modly/blob/main/tools/modly-cli/agent.py)** with helpers `_request_json`, `_multipart_form`, and `_poll_workflow_run`
- The CLI can **auto-start** the backend if not running, using `cmd_serve` → `_start_backend`
- **File uploads** use multipart/form-data construction for image-to-mesh and workflow operations
- **Job polling** queries status endpoints until completion, with optional progress streaming
- **Recovery metadata** in every response enables reproducible automation
- **Electron IPC is not used** by the CLI; the same HTTP API serves both UI and programmatic clients

---

## Frequently Asked Questions

### What port does Modly CLI use to communicate with the desktop app?

By default, the Modly CLI agent connects to `http://127.0.0.1:8765`. This port is configured in `_resolve_serve_config` and can be overridden via environment variables or CLI arguments when starting the backend with `modly-cli serve`.

### Can I use the Modly CLI without opening the desktop GUI?

Yes. Run `modly-cli serve --detach` to start only the FastAPI backend without the Electron frontend. The CLI will then communicate with this headless server exactly as it would with the full desktop application.

### How does the CLI handle large file uploads?

The CLI uses `urllib.request` with `_multipart_form` to construct multipart/form-data payloads. These are streamed to endpoints like `/generate/from-image` or `/workflow-runs/from-image`. The implementation in [`agent.py`](https://github.com/lightningpixel/modly/blob/main/agent.py) handles proper boundary encoding and content-type headers without loading entire files into memory unnecessarily.

### Why doesn't the CLI use Electron's IPC channels?

Electron IPC requires running within the Electron process context. By using HTTP, the Modly CLI maintains language-agnostic compatibility—any HTTP client can control Modly—and supports headless automation. The IPC system in [`electron/main/ipc-handlers.ts`](https://github.com/lightningpixel/modly/blob/main/electron/main/ipc-handlers.ts) remains dedicated to UI-specific operations like window management and native dialogs.