How the Modly CLI Agent Communicates with a Running Desktop App: HTTP API Architecture Explained

The Modly CLI agent communicates with the running desktop app by sending HTTP requests to a FastAPI server embedded in the Electron-based desktop client, using a REST-style API at http://127.0.0.1:8765 by default.

This design allows the CLI to control the same backend that powers the desktop GUI, enabling automation and scripting without requiring the UI to be open. Understanding this communication architecture is essential for building custom workflows, debugging connectivity issues, or extending Modly's capabilities.

---

Key Communication Architecture

The Modly desktop application is built with Electron and embeds a FastAPI server that exposes REST endpoints such as /health, /model/status, and /generate/.... The CLI agent in tools/modly-cli/agent.py interacts with this server through standard HTTP requests—not through Electron's IPC channels.

This separation provides two major benefits:

• Language-agnostic integration — Any tool that can make HTTP requests can control Modly
• UI independence — The backend can run in "headless" mode without the Electron frontend

---

How the CLI Agent Establishes Connection

The communication flow follows a consistent pattern across all CLI operations:

1. Server Discovery and Startup

If the FastAPI server is not running, the CLI can launch it automatically. The cmd_serve function (lines 409–420 in agent.py) resolves configuration via _resolve_serve_config and starts the backend with _start_backend:

# From tools/modly-cli/agent.py

def cmd_serve(args):
    config = _resolve_serve_config(args)
    if args.detach:
        proc = _start_backend(config)  # Spawns uvicorn process

        return {"ok": True, "pid": proc.pid, "base_url": config["base_url"]}

This spawns a Python process running uvicorn main:app, binding to 127.0.0.1:8765 by default.

2. Health Verification

Before executing commands, the CLI validates server availability through _try_health and _require_health:

$ modly-cli health
{
  "ok": true,
  "base_url": "http://127.0.0.1:8765",
  "health": { "status": "ready", "version": "1.2.3" }
}

3. API Request Execution

All functional commands use the _request_json helper (lines 66–84 in agent.py), which wraps Python's urllib.request:

def _request_json(method, path, body=None, headers=None, timeout=DEFAULT_TIMEOUT_SECONDS):
    url = f"{BASE_URL}{path}"
    req = urllib.request.Request(url, method=method, data=body, headers=headers)
    with urllib.request.urlopen(req, timeout=timeout) as response:
        return json.loads(response.read())

This pattern supports GET, POST, PUT, and DELETE operations against FastAPI endpoints defined in api/routers/*.py.

---

Handling File Uploads and Streaming Data

For commands requiring file inputs, the CLI constructs multipart/form-data requests using _multipart_form (lines 115–123 in agent.py):

def _multipart_form(fields, files):
    # fields: dict of form field name → value

    # files: dict of field name → (filename, file_bytes, content_type)

    body, content_type = encode_multipart_formdata(fields, files)
    return body, {"Content-Type": content_type}

These uploads target endpoints like:

• POST /generate/from-image — Image-to-mesh generation
• POST /workflow-runs/from-image — Workflow execution with image input

Example CLI invocation:

$ modly-cli generate --image ./portrait.jpg --output ./portrait.glb
{
  "ok": true,
  "base_url": "http://127.0.0.1:8765",
  "run": { "kind": "workflowRun", "id": "12345" },
  "status": { "status": "done" },
  "workspace_path": "workspace/Agent/portrait.glb",
  "meta": { "status_command": "modly-cli workflow-run status 12345" }
}

---

Polling and Job Progress Monitoring

Long-running operations use a polling loop to track completion. The _poll_workflow_run function (lines 735–751 in agent.py) repeatedly queries status endpoints:

def _poll_workflow_run(run_id, poll_interval=1.0, timeout=None):
    start = time.time()
    while True:
        status = _request_json("GET", f"/workflow-runs/{run_id}")
        if status["status"] in ("done", "failed", "cancelled"):
            return status
        if timeout and (time.time() - start) > timeout:
            raise TimeoutError(f"Polling exceeded {timeout}s")
        time.sleep(poll_interval)
        # Progress optionally emitted to stderr

Equivalent manual status check:

$ modly-cli generate status --job-id 7f9c3a
{
  "ok": true,
  "job_id": "7f9c3a",
  "status": { "status": "running", "progress": 45, "stage": "mesh_reconstruction" }
}

---

Recovery Metadata and Reproducibility

Every response includes a meta object generated by _recovery_meta. This contains exact CLI commands to reproduce or inspect the operation:

{
  "meta": {
    "status_command": "modly-cli workflow-run status 12345",
    "cancel_command": "modly-cli workflow-run cancel 12345",
    "logs_command": "modly-cli workflow-run logs 12345"
  }
}

This self-documenting design enables script automation and debugging without manual URL construction.

---

What the CLI Does NOT Use: Electron IPC

The CLI intentionally bypasses Electron's IPC system. The desktop UI uses separate channels defined in:

• electron/main/ipc-handlers.ts — Main-process IPC handlers
• electron/preload/electron-api.ts — Renderer-to-main bridge

These are UI-only and irrelevant to CLI communication. The FastAPI server serves both the Electron frontend and external HTTP clients uniformly.

---

Running the Backend Without the Desktop UI

For server environments or automation pipelines, start the backend directly:

$ modly-cli serve --detach --print-command
{
  "ok": true,
  "cmd": ["/path/to/python", "-m", "uvicorn", "main:app", "--host", "127.0.0.1", "--port", "8765"],
  "cwd": "/path/to/modly/api",
  "env": { "MODELS_DIR": "/models", "WORKSPACE_DIR": "/workspace" },
  "base_url": "http://127.0.0.1:8765"
}

This produces a headless Modly instance controllable entirely via HTTP.

---

Summary

• The Modly CLI agent communicates through standard HTTP requests to a FastAPI server running inside the Electron desktop app
• Default base URL is http://127.0.0.1:8765, configurable via environment or CLI flags
• Core communication code lives in tools/modly-cli/agent.py with helpers _request_json, _multipart_form, and _poll_workflow_run
• The CLI can auto-start the backend if not running, using cmd_serve → _start_backend
• File uploads use multipart/form-data construction for image-to-mesh and workflow operations
• Job polling queries status endpoints until completion, with optional progress streaming
• Recovery metadata in every response enables reproducible automation
• Electron IPC is not used by the CLI; the same HTTP API serves both UI and programmatic clients

---

Frequently Asked Questions

What port does Modly CLI use to communicate with the desktop app?

By default, the Modly CLI agent connects to http://127.0.0.1:8765. This port is configured in _resolve_serve_config and can be overridden via environment variables or CLI arguments when starting the backend with modly-cli serve.

Can I use the Modly CLI without opening the desktop GUI?

Yes. Run modly-cli serve --detach to start only the FastAPI backend without the Electron frontend. The CLI will then communicate with this headless server exactly as it would with the full desktop application.

How does the CLI handle large file uploads?

The CLI uses urllib.request with _multipart_form to construct multipart/form-data payloads. These are streamed to endpoints like /generate/from-image or /workflow-runs/from-image. The implementation in agent.py handles proper boundary encoding and content-type headers without loading entire files into memory unnecessarily.

Why doesn't the CLI use Electron's IPC channels?

Electron IPC requires running within the Electron process context. By using HTTP, the Modly CLI maintains language-agnostic compatibility—any HTTP client can control Modly—and supports headless automation. The IPC system in electron/main/ipc-handlers.ts remains dedicated to UI-specific operations like window management and native dialogs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}