How the modly CLI Interfaces with a Running Desktop App via Canonical Commands

The modly CLI communicates with the running Electron desktop app by sending HTTP requests to a local FastAPI server (the Modly API) that exposes canonical JSON-first commands for automation agents.

The modly project's command-line interface (modly-cli) provides agent-friendly automation without direct Electron process manipulation. Instead, it leverages a REST API contract that both the desktop UI and CLI share. Below is a complete technical breakdown of this architecture as implemented in lightningpixel/modly.

Architecture Overview: FastAPI as the Bridge

The Modly desktop application runs an internal FastAPI server on http://127.0.0.1:8765 while the Electron UI is active. According to the source code in src/shared/hooks/useApi.ts, the React front-end consumes this same apiUrl—demonstrating that the CLI and GUI are first-class consumers of identical endpoints.

The CLI implementation in tools/modly-cli/agent.py contains no Electron-specific code. It is a std-lib-only HTTP client (using urllib.request) that treats the desktop app as a headless service.

Step 1: API Discovery and Health Verification

Before executing any canonical command, the CLI locates the server and validates connectivity.

From tools/modly-cli/agent.py line 40:

DEFAULT_BASE_URL = os.environ.get("MODLY_API_URL", "http://127.0.0.1:8765")

The _require_health helper performs an initial GET /health check:

def _require_health(base_url, timeout):
    return _request_json("GET", f"{base_url.rstrip('/')}/health", timeout=timeout)

This ensures the desktop app is running and responsive before proceeding with operations that depend on GPU resources or loaded models.

Step 2: Canonical Command Structure

The CLI organizes functionality into canonical verbs implemented as argparse sub-commands. Each maps to a dedicated cmd_* function that follows a consistent pattern:

1. Resolve base_url from arguments or environment
2. Call _require_health() for readiness
3. Build the REST URL and execute via _request_json()
4. Wrap the response with _recovery_meta() and output via _json_print()

The canonical command surface includes:

Command	API Endpoint	Purpose
health	GET /health	Liveness probe
model list	GET /model/all	Enumerate available ML models
workflow-run start	POST /workflow-runs/from-image	Begin processing with image upload
workflow-run status	GET /workflow-runs/{id}	Poll run state
workflow-run cancel	POST /workflow-runs/{id}/cancel	Abort running operation
capability	GET /capability/*	Query feature flags
process-run	Various /process-runs/*	Lower-level execution primitives

The SKILL.md specification in tools/modly-cli/ documents this contract for human and agent consumers.

Step 3: Workflow-Run Execution with Polling

The workflow-run start command demonstrates multipart file upload and synchronous completion handling. From agent.py lines 126-130:

# Polling loop excerpt

while True:
    status_data = _request_json("GET", f"{base_url}/workflow-runs/{run_id}")
    if status_data.get("status") == "done":
        break
    time.sleep(poll_interval)

The server implementation in src/areas/workflows/workflowRunStore.ts handles these endpoints, persisting run state that survives UI restarts.

Step 4: High-Level Generation Shortcut

The generate command (lines 96-106) orchestrates the full pipeline:

def cmd_generate(args):
    # 1. Resolve model (auto|active|explicit ID)

    # 2. Upload via workflow-run

    # 3. Poll to completion

    # 4. Call /export/{fmt} for mesh download

    # 5. Return unified JSON with paths and recovery commands

This convenience wrapper collapses multiple API calls into a single agent-friendly operation while preserving full introspection through the returned metadata.

Step 5: Recovery Metadata for Stateless Automation

Every mutating command embeds recovery commands in its JSON output via _recovery_meta() (lines 202-210):

{
  "ok": true,
  "run": {"kind": "workflowRun", "id": "wr_abc123"},
  "meta": {
    "status_command": "python tools/modly-cli/agent.py workflow-run status wr_abc123",
    "cancel_command": "python tools/modly-cli/agent.py workflow-run cancel wr_abc123",
    "canonical": "workflow-run start"
  }
}

This design makes automation fully reproducible and resumable without external state management. An agent that receives a timeout or interruption can always recover using the embedded CLI commands.

Error Handling: Guaranteed JSON Output

All failures raise ModlyCliError (lines 48-56), caught at the top-level main() and serialized as:

{"ok": false, "code": "HEALTH_TIMEOUT", "message": "..."}

This contract ensures parsers never encounter unstructured stderr output.

Practical Usage Examples

Check desktop app availability:

python tools/modly-cli/agent.py health

List available models:

python tools/modly-cli/agent.py model list

Start a workflow with blocking completion:

python tools/modly-cli/agent.py workflow-run start \
    --image ./photo.png \
    --wait \
    --output ./result.glb

Direct mesh generation with progress:

python tools/modly-cli/agent.py generate \
    --image ./object.png \
    --output ./mesh.glb \
    --progress

Resume monitoring using recovery metadata:

python tools/modly-cli/agent.py workflow-run status <run_id>

Key Source Files

• tools/modly-cli/agent.py — CLI implementation with HTTP helpers, canonical command dispatch, and recovery metadata generation
• tools/modly-cli/SKILL.md — Canonical command specification
• src/shared/hooks/useApi.ts — React hook showing shared API URL contract
• src/areas/workflows/workflowRunStore.ts — Server-side workflow run persistence

Summary

• The modly CLI never interacts directly with Electron processes; it communicates via HTTP to the FastAPI server (http://127.0.0.1:8765) spawned by the desktop app
• Environment variable MODLY_API_URL allows custom endpoints for testing or remote scenarios
• Canonical commands (health, model, workflow-run, capability, process-run, generate) provide a stable, versioned automation surface
• Recovery metadata embedded in every response enables stateless, resumable automation without external orchestration
• JSON-only output with guaranteed error structure supports reliable programmatic parsing

Frequently Asked Questions

What port does the modly CLI use to connect to the desktop app?

The CLI defaults to port 8765 on 127.0.0.1, reading from the MODLY_API_URL environment variable if set otherwise. This is defined in tools/modly-cli/agent.py line 40.

Can I use the modly CLI while the desktop GUI is closed?

No. The CLI requires the Electron app to be running because the FastAPI server is launched as part of the desktop application's lifecycle. The /health endpoint will fail if the GUI process has exited.

What makes the commands "canonical"?

Canonical commands follow a strict JSON-first contract documented in SKILL.md: consistent parameter naming, deterministic output structure, embedded recovery commands, and guaranteed error serialization. This design prioritizes agent interoperability over human ergonomics.

How do I resume a workflow that was interrupted?

Extract the status_command or cancel_command from the meta object of any prior CLI response, or manually invoke python tools/modly-cli/agent.py workflow-run status <run_id> using the run ID returned from the start operation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}