How to Run Tests Against the cmux Socket API: A Complete Guide

To run tests against the cmux socket API, launch a tagged Debug build with -socketControlMode allowAll, verify the Unix socket at /tmp/cmux-debug-<tag>.sock, and execute the Python test suite from tests_v2/ with the CMUX_SOCKET environment variable set.

The manaflow-ai/cmux repository exposes a Unix-domain socket API that provides external programs with a deterministic endpoint to inspect or mutate the state of a running cmux instance. This architecture enables comprehensive end-to-end testing through both Swift UI tests and a Python-based test harness, validating everything from simple health checks to complex workspace-relative routing and focus management policies.

Understanding the Socket API Architecture

The cmux socket API is implemented across several core components that handle listener initialization, command validation, and security policy enforcement.

Core Components

Component	Role	Source File
TerminalController	Starts the Unix-socket listener, validates incoming commands, and enforces the socket-control policy (focus-mutation allowance, authentication, and mode handling).	Sources/TerminalController.swift
SocketControlSettings	Stores user-visible settings that determine if the socket is disabled, password-protected, or fully open (allowAll). Provides the default stable socket path used by test suites.	Sources/SocketControlSettings.swift
Workspace.roundTripUnixSocket	Low-level helper that creates a client socket, writes a request, reads the JSON response, and translates errors into cmuxError. Used by both the CLI and Python test harness.	Sources/Workspace.swift (lines 2942-2980)
AutomationSocketUITests	Swift-based UI test suite that validates socket health, ping/pong behavior, and command handling inside the app. Serves as an in-process client example.	cmuxUITests/AutomationSocketUITests.swift
tests_v2/*.py	End-to-end Python test suite that drives a real cmux binary via the socket, checking JSON output, workspace-relative routing, and notifications.	tests_v2/test_workspace_relative.py

Socket Lifecycle and Policy Enforcement

When you launch a tagged Debug build using ./scripts/reload.sh --tag socket-test --launch, the script injects the launch argument -socketControlMode allowAll. This tells TerminalController to create a listener using socket(AF_UNIX, SOCK_STREAM, 0) on the path returned by SocketControlSettings.stableDefaultSocketPath, typically /tmp/cmux-debug-<tag>.sock.

A lock-protected depth counter, socketCommandPolicyDepth, tracks whether focus-mutating commands are permitted. Only commands that explicitly allow focus mutations—such as those prefixed with focus_allow true—may call focus-changing APIs, ensuring tests do not inadvertently steal macOS focus.

Launching a Socket-Enabled cmux Instance

Before running tests, you must start a cmux instance that exposes its control socket with open access.

Build and launch a tagged Debug app isolated from existing instances:

./scripts/reload.sh --tag socket-test --launch

The script prints an App path: line showing the location of the built binary. The socket file is created automatically under /tmp using the tag name, e.g., /tmp/cmux-debug-socket-test.sock.

Verify the socket is responsive using the CLI binary from the derived-data directory:

SOCKET=/tmp/cmux-debug-socket-test.sock
DERIVED="$HOME/Library/Developer/Xcode/DerivedData/cmux-socket-test/Build/Products/Debug"
"$DERIVED/cmux" --socket "$SOCKET" ping

# → PONG

Running the Socket API Test Suite

Once the socket is active, you can execute the multi-layered test suite.

Python End-to-End Tests (tests_v2)

The tests_v2 directory contains the primary Python test harness. Set the environment variable so the harness can locate the socket:

export CMUX_SOCKET="/tmp/cmux-debug-socket-test.sock"

# Legacy fallback also supported

export CMUX_SOCKET_PATH="/tmp/cmux-debug-socket-test.sock"

Execute the suite using pytest:

cd tests_v2
python3 -m pytest -vv

The harness automatically discovers the most recent cmux binary via the _find_cli_binary logic in each test file. Tests construct CLI commands like cmux --socket <path> … and verify JSON responses, workspace-relative routing, and notification delivery.

Swift UI Tests (AutomationSocketUITests.swift)

For in-process validation, the Swift UI tests use waitForSocketPong to confirm socket health before executing commands. These tests run inside the Xcode test runner and validate that TerminalController correctly marshals errors back as "ERROR <msg>" strings.

Manual CLI Verification

You can replicate test scenarios manually to debug specific behaviors. For example, to list panels in a specific workspace:

./cmux --socket /tmp/cmux-debug-socket-test.sock \
    list-panels --workspace "$CMUX_WORKSPACE_ID"

Common Test Patterns and Commands

Effective cmux socket API testing relies on several canonical interaction patterns.

• Health Verification: socketCommand("ping") must return "PONG". This confirms the socket listener is active in TerminalController.
• Workspace-Relative Routing: Set CMUX_WORKSPACE_ID or pass --workspace <ref> to ensure the server handles the request in the intended workspace context, as validated in test_workspace_relative.py.
• Focus-Mutation Permissions: Prepend focus_allow true to command chains when testing focus-changing operations. The policy stack depth must be non-zero, or TerminalController rejects the command to prevent focus stealing.

Example of granting focus permission:

./cmux --socket "$SOCKET" \
    "focus_allow true; focus_window $(cat /tmp/window-id)"

Troubleshooting Common Pitfalls

Symptom	Likely Cause	Solution
socketCommand("ping") returns "failed to connect"	Socket not started; app launched with default socketControlMode = cmuxOnly.	Relaunch with -socketControlMode allowAll via ./scripts/reload.sh.
Python tests cannot locate the CLI binary	Derived-data path changed after clean build.	Re-run ./scripts/reload.sh --tag socket-test so _find_cli_binary selects the newest executable.
"ERROR socket path mismatch"	CMUX_SOCKET_PATH points to stale socket from previous run.	Kill the stale app (pkill cmux) and delete /tmp/cmux-*.sock files.
Focus-changing commands rejected	socketCommandPolicyDepth is zero (no focus_allow true).	Prepend focus_allow true to the command chain or use the UI test helper that manages the policy stack.

Complete Code Examples

Launching and Verifying a Tagged Build

# Build and launch

./scripts/reload.sh --tag socket-test --launch

# Verify socket path matches the tag

ls -la /tmp/cmux-debug-socket-test.sock

Python Test Harness Setup

import os, json, subprocess

socket_path = os.getenv("CMUX_SOCKET", "/tmp/cmux-debug-socket-test.sock")
cli_path = "/path/to/derivedData/Build/Products/Debug/cmux"

def socket_command(*args):
    cmd = [cli_path, "--socket", socket_path] + list(args)
    result = subprocess.run(cmd, capture_output=True, text=True, check=True)
    return json.loads(result.stdout)

# Example: Get workspace surfaces

workspace_id = os.getenv("CMUX_WORKSPACE_ID")
data = socket_command("list-panels", "--workspace", workspace_id)
print(data["surfaces"])

Running the Full Test Suite with Environment Variables

# Ensure the tagged app is running with socket exposed

./scripts/reload.sh --tag socket-test --launch

# Export socket location for the Python harness

export CMUX_SOCKET=/tmp/cmux-debug-socket-test.sock

# Execute

cd tests_v2 && python3 -m pytest -vv test_workspace_relative.py

Summary

• Launch tagged Debug builds with -socketControlMode allowAll to expose the Unix socket at /tmp/cmux-debug-<tag>.sock for testing.
• The TerminalController enforces command policies via socketCommandPolicyDepth, requiring focus_allow true for focus-mutating operations.
• Use CMUX_SOCKET to point the Python tests_v2 suite at the running instance.
• Reference AutomationSocketUITests.swift for in-process Swift testing patterns and Workspace.swift (lines 2942-2980) for low-level socket round-trip implementation details.
• Clear stale socket files and kill existing cmux processes when encountering connection errors.

Frequently Asked Questions

Why does my test fail to connect to the cmux socket?

The application likely launched with the default socketControlMode set to cmuxOnly, which disables external socket access. You must launch the app using ./scripts/reload.sh --tag <name> --launch to inject the -socketControlMode allowAll argument, which tells TerminalController to start the listener.

How do I enable focus-changing commands in my socket API tests?

Focus-mutating commands like focus_window require a non-zero policy depth. Send the command focus_allow true before the focus operation, e.g., "focus_allow true; focus_window <id>". This increments the lock-protected socketCommandPolicyDepth counter in TerminalController, allowing the mutation while preventing the test from stealing system focus.

Where is the socket file created during testing?

By default, SocketControlSettings.stableDefaultSocketPath generates a path under /tmp using the launch tag, formatted as /tmp/cmux-debug-<tag>.sock. When using ./scripts/reload.sh --tag socket-test, the socket appears at /tmp/cmux-debug-socket-test.sock.

Can I run the socket API tests against a Release build?

No. The socket API test suite is designed for Debug builds launched via ./scripts/reload.sh with specific launch arguments. Release builds typically run with restricted socketControlMode settings for security, and the Python harness relies on the predictable derived-data paths and socket locations established by the reload script's tagging system.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}