# How to Programmatically Control vphone-cli Host via Unix Domain Socket

> Programmatically control vphone-cli host via Unix domain socket. Send JSON commands for screenshots, touch input, key presses, and clipboard operations from local processes.

- Repository: [Lakr/vphone-cli](https://github.com/Lakr233/vphone-cli)
- Tags: how-to-guide
- Published: 2026-09-09

---

**vphone-cli exposes a Unix-domain socket server that accepts JSON commands for screenshots, touch input, key presses, and clipboard operations from any local process.**

The [Lakr233/vphone-cli](https://github.com/Lakr233/vphone-cli) repository implements a lightweight control interface for virtualized phone environments on macOS. When the application launches, it starts a **Unix-domain socket** server that enables external programs to programmatically control the guest VM without modifying internal virtualization layers. This mechanism allows developers to build automation workflows, testing frameworks, and remote control interfaces using simple JSON-over-socket communication.

## Socket Architecture and Initialization

### Runtime Socket Creation

When `VPhoneAppDelegate` finishes launching, it constructs the socket path adjacent to the VM configuration file and instantiates the control server. In [`sources/vphone-cli/VPhoneAppDelegate.swift`](https://github.com/Lakr233/vphone-cli/blob/main/sources/vphone-cli/VPhoneAppDelegate.swift) (lines 55‑66), the implementation generates the socket path and launches the server:

```swift
let socketPath = options.configURL
    .deletingLastPathComponent()
    .appendingPathComponent("vphone.sock").path
let hc = VPhoneHostControl(socketPath: socketPath)
hc.start(captureView: wc.captureView!,
         screenRecorder: recorder,
         control: control,
         screenWidth: options.screenWidth,
         screenHeight: options.screenHeight)

```

The socket typically resides in the same directory as the VM configuration, commonly `/tmp/vphone.sock` or a path relative to the bundle. The `start` method injects three critical dependencies: `captureView` for touch injection, `screenRecorder` for screenshots, and `control` for HID/clipboard operations.

### Server Implementation Details

The `VPhoneHostControl` class in [`sources/vphone-cli/VPhoneHostControl.swift`](https://github.com/Lakr233/vphone-cli/blob/main/sources/vphone-cli/VPhoneHostControl.swift) implements a standard Berkeley socket server using `AF_UNIX`. The implementation follows this sequence:

- **Socket setup**: Creates a `SOCK_STREAM` socket, binds to the filesystem path, and begins listening.
- **Accept loop**: The `acceptLoop` (lines 20‑26) runs indefinitely on a background dispatch queue, accepting client connections via `accept()`.
- **Client handling**: Each connection spawns `handleClient` (lines 28‑122), which reads a single newline-terminated JSON command, executes the action on the main actor, and writes a JSON response before closing the connection.

The protocol mandates **single-line JSON messages** terminated by a newline character (`\n`). The server processes one command per connection, making it stateless and safe for concurrent scripted access.

## JSON Command Protocol

The socket API recognizes five primary command types, distinguished by the `"t"` field in the JSON payload. As documented in [`VPhoneHostControl.swift`](https://github.com/Lakr233/vphone-cli/blob/main/VPhoneHostControl.swift) (lines 14‑21), the supported operations are:

- **`"screenshot"`** – Captures the current display. Optionally accepts a `"path"` string to save a full-resolution PNG. Always returns a compact base64-encoded grayscale JPEG in the response.
- **`"tap"`** – Simulates a single touch at screen coordinates. Requires `"x"` and `"y"` integer parameters in pixels.
- **`"swipe"`** – Performs a drag gesture between two points. Requires `"x1"`, `"y1"`, `"x2"`, and `"y2"` coordinates. Optionally accepts `"ms"` to specify gesture duration in milliseconds (defaults to system standard).
- **`"key"`** – Sends a HID key event. The `"name"` field accepts: `home`, `power`, `volup`, or `voldown`.
- **`"type"`** – Sets the guest clipboard to a specified string via the `"text"` field, effectively simulating text input.

## Response Format

Every command receives a single-line JSON response written by `writeResponse` (lines 33‑45). The response object always contains:

- **`ok`**: Boolean indicating success or failure.
- **`path`**: (Optional) Filesystem path when `"screenshot"` includes a save location.
- **`error`**: (Optional) Error message string when `ok` is `false`.
- **`image`**: (Optional) Compact base64-encoded grayscale JPEG image, approximately 1/9th the size of the original screenshot, useful for visual verification without file I/O.

## Practical Code Examples

### Shell Commands with Netcat

For quick automation or shell scripts, use `nc` (netcat) to send JSON commands to the socket:

```bash

# Define the socket path (check console output at startup for exact location)

SOCK=/tmp/vphone.sock

# Simulate a tap at coordinate (645, 1398)

echo '{"t":"tap","x":645,"y":1398}' | nc -U $SOCK

# Perform a swipe from bottom to top over 300ms

echo '{"t":"swipe","x1":645,"y1":2600,"x2":645,"y2":1400,"ms":300}' | nc -U $SOCK

# Press the Home button

echo '{"t":"key","name":"home"}' | nc -U $SOCK

# Save a full-resolution screenshot and receive compact preview

echo '{"t":"screenshot","path":"/tmp/shot.png"}' | nc -U $SOCK

# Inject text into the guest clipboard

echo '{"t":"type","text":"Hello from the host!"}' | nc -U $SOCK

```

Each command outputs a JSON response line, such as:

```json
{"ok":true,"path":"/tmp/shot.png","image":"<base64-encoded-grayscale-jpeg>"}

```

### Python Client Implementation

For programmatic automation, Python's `socket` module provides a robust client:

```python
#!/usr/bin/env python3
import socket
import json
import base64
from pathlib import Path

SOCK_PATH = "/tmp/vphone.sock"  # Adjust to match your configuration

def send_cmd(cmd: dict) -> dict:
    """Send a JSON command and return the parsed response."""
    with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as s:
        s.connect(SOCK_PATH)
        # Commands must be terminated by newline

        s.sendall(json.dumps(cmd).encode() + b'\n')
        # Read single-line response

        resp = s.recv(8192).decode().strip()
    return json.loads(resp)

# Capture compact screenshot without saving file

resp = send_cmd({"t": "screenshot"})
if resp["ok"]:
    img_data = base64.b64decode(resp["image"])
    Path("preview.jpg").write_bytes(img_data)
    print("Screenshot captured")
else:
    print("Error:", resp.get("error"))

# Tap and capture result state

resp = send_cmd({"t": "tap", "x": 645, "y": 1398})
if resp["ok"]:
    img = base64.b64decode(resp["image"])
    Path("after_tap.jpg").write_bytes(img)

```

### Swift Native Client

For macOS applications integrating with **vphone-cli**, use Foundation's `socket` API:

```swift
import Foundation

func send(command: [String: Any], to socketPath: String) throws -> [String: Any] {
    let fd = socket(AF_UNIX, SOCK_STREAM, 0)
    defer { close(fd) }
    
    var addr = sockaddr_un()
    addr.sun_family = sa_family_t(AF_UNIX)
    let pathBytes = socketPath.utf8CString
    withUnsafeMutablePointer(to: &addr.sun_path) { ptr in
        ptr.withMemoryRebound(to: CChar.self, capacity: pathBytes.count) { dst in
            for (i, b) in pathBytes.enumerated() { dst[i] = b }
        }
    }
    
    let len = socklen_t(MemoryLayout<sockaddr_un>.size)
    try connect(fd, withUnsafePointer(to: &addr) { 
        $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { $0 } 
    }, len)
    
    let data = try JSONSerialization.data(withJSONObject: command)
    var payload = data + [0x0A] // newline
    
    try payload.withUnsafeBytes { ptr in
        var sent = 0
        while sent < payload.count {
            let n = write(fd, ptr.baseAddress! + sent, payload.count - sent)
            if n <= 0 { throw NSError(domain: "SocketError", code: -1) }
            sent += n
        }
    }
    
    var buffer = [UInt8](repeating: 0, count: 4096)
    var response = Data()
    while true {
        let n = read(fd, &buffer, buffer.count)
        if n <= 0 { break }
        response.append(contentsOf: buffer[0..<n])
        if buffer[0..<n].contains(0x0A) { break }
    }
    
    return try JSONSerialization.jsonObject(with: response) as! [String: Any]
}

// Usage example
let result = try send(command: ["t": "key", "name": "power"], to: "/tmp/vphone.sock")
print(result)

```

## Key Implementation Files

Understanding the source architecture helps when extending functionality or debugging socket interactions:

- **[`sources/vphone-cli/VPhoneHostControl.swift`](https://github.com/Lakr233/vphone-cli/blob/main/sources/vphone-cli/VPhoneHostControl.swift)** – Contains the complete socket server implementation, including `acceptLoop`, `handleClient`, and `writeResponse`.
- **[`sources/vphone-cli/VPhoneAppDelegate.swift`](https://github.com/Lakr233/vphone-cli/blob/main/sources/vphone-cli/VPhoneAppDelegate.swift)** – Initializes the socket path and starts the control server during application launch.
- **[`sources/vphone-cli/VPhoneControl.swift`](https://github.com/Lakr233/vphone-cli/blob/main/sources/vphone-cli/VPhoneControl.swift)** – Provides the underlying HID event injection and clipboard APIs used by the socket commands.
- **[`sources/vphone-cli/VPhoneScreenRecorder.swift`](https://github.com/Lakr233/vphone-cli/blob/main/sources/vphone-cli/VPhoneScreenRecorder.swift)** – Implements the private API screenshot capture that generates both full-resolution files and compact JPEG previews.
- **[`sources/vphone-cli/VPhoneVirtualMachineView.swift`](https://github.com/Lakr233/vphone-cli/blob/main/sources/vphone-cli/VPhoneVirtualMachineView.swift)** – Declares the touch injection interface required for `tap` and `swipe` commands.

## Summary

- **vphone-cli** creates a **Unix-domain socket** at runtime, typically located beside the VM configuration file.
- The socket accepts **single-line JSON commands** via TCP-like stream semantics, supporting `screenshot`, `tap`, `swipe`, `key`, and `type` operations.
- Responses always include an `ok` boolean and may contain `image` (base64 compact JPEG), `path`, or `error` fields.
- The architecture uses three injected services: `VPhoneVirtualMachineView` for input, `VPhoneScreenRecorder` for capture, and `VPhoneControl` for HID/clipboard.
- Clients can connect using standard tools like **netcat**, or implement native clients in **Python**, **Swift**, or any language supporting Unix sockets.

## Frequently Asked Questions

### Where is the vphone-cli Unix socket located?

The socket path is derived from the VM configuration file location at runtime. In [`VPhoneAppDelegate.swift`](https://github.com/Lakr233/vphone-cli/blob/main/VPhoneAppDelegate.swift) (lines 55‑66), the code constructs the path by replacing the configuration filename with `vphone.sock`. If your config resides at `~/VMs/phone.cfg`, expect the socket at `~/VMs/vphone.sock`. Check the application console output at startup for the exact path.

### What automation commands does vphone-cli support?

The socket API supports five command types: **`screenshot`** (capture display), **`tap`** (touch at x,y), **`swipe`** (drag between coordinates), **`key`** (hardware buttons like home/power/volume), and **`type`** (clipboard text injection). Each command requires a `"t"` field specifying the type and relevant payload parameters.

### How do I capture screenshots via the socket API?

Send `{"t":"screenshot"}` to receive a compact base64-encoded grayscale JPEG in the `image` field of the response. To save a full-resolution PNG to disk, include the optional `"path"` parameter, such as `{"t":"screenshot","path":"/tmp/capture.png"}`. The response will include both the `path` and the compact `image` preview.

### Can multiple clients connect to the socket simultaneously?

Yes, but each connection handles exactly one command before closing. The `acceptLoop` in [`VPhoneHostControl.swift`](https://github.com/Lakr233/vphone-cli/blob/main/VPhoneHostControl.swift) spawns concurrent `handleClient` tasks on a background queue, allowing multiple scripts or applications to send commands simultaneously. However, the VM actions themselves execute serially on the main actor to prevent race conditions in the guest state.