# How the modly CLI Interfaces with a Running Desktop App via Canonical Commands

> Discover how the modly CLI interacts with your desktop app using canonical commands. Learn about the FastAPI server and JSON-first automation for seamless integration.

- Repository: [lightningpixel/modly](https://github.com/lightningpixel/modly)
- Tags: internals
- Published: 2026-08-15

---

**The modly CLI communicates with the running Electron desktop app by sending HTTP requests to a local FastAPI server (the Modly API) that exposes canonical JSON-first commands for automation agents.**

The **modly** project's command-line interface (`modly-cli`) provides agent-friendly automation without direct Electron process manipulation. Instead, it leverages a REST API contract that both the desktop UI and CLI share. Below is a complete technical breakdown of this architecture as implemented in `lightningpixel/modly`.

## Architecture Overview: FastAPI as the Bridge

The Modly desktop application runs an internal **FastAPI server** on `http://127.0.0.1:8765` while the Electron UI is active. According to the source code in [`src/shared/hooks/useApi.ts`](https://github.com/lightningpixel/modly/blob/main/src/shared/hooks/useApi.ts), the React front-end consumes this same `apiUrl`—demonstrating that the CLI and GUI are first-class consumers of identical endpoints.

The CLI implementation in [`tools/modly-cli/agent.py`](https://github.com/lightningpixel/modly/blob/main/tools/modly-cli/agent.py) contains no Electron-specific code. It is a **std-lib-only HTTP client** (using `urllib.request`) that treats the desktop app as a headless service.

## Step 1: API Discovery and Health Verification

Before executing any canonical command, the CLI locates the server and validates connectivity.

From [`tools/modly-cli/agent.py`](https://github.com/lightningpixel/modly/blob/main/tools/modly-cli/agent.py) line 40:

```python
DEFAULT_BASE_URL = os.environ.get("MODLY_API_URL", "http://127.0.0.1:8765")

```

The `_require_health` helper performs an initial `GET /health` check:

```python
def _require_health(base_url, timeout):
    return _request_json("GET", f"{base_url.rstrip('/')}/health", timeout=timeout)

```

This ensures the desktop app is running and responsive before proceeding with operations that depend on GPU resources or loaded models.

## Step 2: Canonical Command Structure

The CLI organizes functionality into **canonical verbs** implemented as `argparse` sub-commands. Each maps to a dedicated `cmd_*` function that follows a consistent pattern:

1. Resolve `base_url` from arguments or environment
2. Call `_require_health()` for readiness
3. Build the REST URL and execute via `_request_json()`
4. Wrap the response with `_recovery_meta()` and output via `_json_print()`

The canonical command surface includes:

| Command | API Endpoint | Purpose |
|---------|--------------|---------|
| `health` | `GET /health` | Liveness probe |
| `model list` | `GET /model/all` | Enumerate available ML models |
| `workflow-run start` | `POST /workflow-runs/from-image` | Begin processing with image upload |
| `workflow-run status` | `GET /workflow-runs/{id}` | Poll run state |
| `workflow-run cancel` | `POST /workflow-runs/{id}/cancel` | Abort running operation |
| `capability` | `GET /capability/*` | Query feature flags |
| `process-run` | Various `/process-runs/*` | Lower-level execution primitives |

The [`SKILL.md`](https://github.com/lightningpixel/modly/blob/main/SKILL.md) specification in `tools/modly-cli/` documents this contract for human and agent consumers.

## Step 3: Workflow-Run Execution with Polling

The `workflow-run start` command demonstrates multipart file upload and synchronous completion handling. From [`agent.py`](https://github.com/lightningpixel/modly/blob/main/agent.py) lines 126-130:

```python

# Polling loop excerpt

while True:
    status_data = _request_json("GET", f"{base_url}/workflow-runs/{run_id}")
    if status_data.get("status") == "done":
        break
    time.sleep(poll_interval)

```

The server implementation in [`src/areas/workflows/workflowRunStore.ts`](https://github.com/lightningpixel/modly/blob/main/src/areas/workflows/workflowRunStore.ts) handles these endpoints, persisting run state that survives UI restarts.

## Step 4: High-Level Generation Shortcut

The `generate` command (lines 96-106) orchestrates the full pipeline:

```python
def cmd_generate(args):
    # 1. Resolve model (auto|active|explicit ID)

    # 2. Upload via workflow-run

    # 3. Poll to completion

    # 4. Call /export/{fmt} for mesh download

    # 5. Return unified JSON with paths and recovery commands

```

This convenience wrapper collapses multiple API calls into a single agent-friendly operation while preserving full introspection through the returned metadata.

## Step 5: Recovery Metadata for Stateless Automation

Every mutating command embeds **recovery commands** in its JSON output via `_recovery_meta()` (lines 202-210):

```json
{
  "ok": true,
  "run": {"kind": "workflowRun", "id": "wr_abc123"},
  "meta": {
    "status_command": "python tools/modly-cli/agent.py workflow-run status wr_abc123",
    "cancel_command": "python tools/modly-cli/agent.py workflow-run cancel wr_abc123",
    "canonical": "workflow-run start"
  }
}

```

This design makes automation **fully reproducible and resumable** without external state management. An agent that receives a timeout or interruption can always recover using the embedded CLI commands.

## Error Handling: Guaranteed JSON Output

All failures raise `ModlyCliError` (lines 48-56), caught at the top-level `main()` and serialized as:

```json
{"ok": false, "code": "HEALTH_TIMEOUT", "message": "..."}

```

This contract ensures parsers never encounter unstructured stderr output.

## Practical Usage Examples

Check desktop app availability:

```bash
python tools/modly-cli/agent.py health

```

List available models:

```bash
python tools/modly-cli/agent.py model list

```

Start a workflow with blocking completion:

```bash
python tools/modly-cli/agent.py workflow-run start \
    --image ./photo.png \
    --wait \
    --output ./result.glb

```

Direct mesh generation with progress:

```bash
python tools/modly-cli/agent.py generate \
    --image ./object.png \
    --output ./mesh.glb \
    --progress

```

Resume monitoring using recovery metadata:

```bash
python tools/modly-cli/agent.py workflow-run status <run_id>

```

## Key Source Files

- **[`tools/modly-cli/agent.py`](https://github.com/lightningpixel/modly/blob/main/tools/modly-cli/agent.py)** — CLI implementation with HTTP helpers, canonical command dispatch, and recovery metadata generation
- **[`tools/modly-cli/SKILL.md`](https://github.com/lightningpixel/modly/blob/main/tools/modly-cli/SKILL.md)** — Canonical command specification
- **[`src/shared/hooks/useApi.ts`](https://github.com/lightningpixel/modly/blob/main/src/shared/hooks/useApi.ts)** — React hook showing shared API URL contract
- **[`src/areas/workflows/workflowRunStore.ts`](https://github.com/lightningpixel/modly/blob/main/src/areas/workflows/workflowRunStore.ts)** — Server-side workflow run persistence

## Summary

- The **modly CLI** never interacts directly with Electron processes; it communicates via HTTP to the **FastAPI server** (`http://127.0.0.1:8765`) spawned by the desktop app
- **Environment variable `MODLY_API_URL`** allows custom endpoints for testing or remote scenarios
- **Canonical commands** (`health`, `model`, `workflow-run`, `capability`, `process-run`, `generate`) provide a stable, versioned automation surface
- **Recovery metadata** embedded in every response enables stateless, resumable automation without external orchestration
- **JSON-only output** with guaranteed error structure supports reliable programmatic parsing

## Frequently Asked Questions

### What port does the modly CLI use to connect to the desktop app?

The CLI defaults to port **8765** on `127.0.0.1`, reading from the `MODLY_API_URL` environment variable if set otherwise. This is defined in [`tools/modly-cli/agent.py`](https://github.com/lightningpixel/modly/blob/main/tools/modly-cli/agent.py) line 40.

### Can I use the modly CLI while the desktop GUI is closed?

**No.** The CLI requires the Electron app to be running because the FastAPI server is launched as part of the desktop application's lifecycle. The `/health` endpoint will fail if the GUI process has exited.

### What makes the commands "canonical"?

Canonical commands follow a strict JSON-first contract documented in [`SKILL.md`](https://github.com/lightningpixel/modly/blob/main/SKILL.md): consistent parameter naming, deterministic output structure, embedded recovery commands, and guaranteed error serialization. This design prioritizes **agent interoperability** over human ergonomics.

### How do I resume a workflow that was interrupted?

Extract the `status_command` or `cancel_command` from the `meta` object of any prior CLI response, or manually invoke `python tools/modly-cli/agent.py workflow-run status <run_id>` using the run ID returned from the start operation.