How to Programmatically Control vphone-cli Host via Unix Domain Socket

vphone-cli exposes a Unix-domain socket server that accepts JSON commands for screenshots, touch input, key presses, and clipboard operations from any local process.

The Lakr233/vphone-cli repository implements a lightweight control interface for virtualized phone environments on macOS. When the application launches, it starts a Unix-domain socket server that enables external programs to programmatically control the guest VM without modifying internal virtualization layers. This mechanism allows developers to build automation workflows, testing frameworks, and remote control interfaces using simple JSON-over-socket communication.

Socket Architecture and Initialization

Runtime Socket Creation

When VPhoneAppDelegate finishes launching, it constructs the socket path adjacent to the VM configuration file and instantiates the control server. In sources/vphone-cli/VPhoneAppDelegate.swift (lines 55‑66), the implementation generates the socket path and launches the server:

let socketPath = options.configURL
    .deletingLastPathComponent()
    .appendingPathComponent("vphone.sock").path
let hc = VPhoneHostControl(socketPath: socketPath)
hc.start(captureView: wc.captureView!,
         screenRecorder: recorder,
         control: control,
         screenWidth: options.screenWidth,
         screenHeight: options.screenHeight)

The socket typically resides in the same directory as the VM configuration, commonly /tmp/vphone.sock or a path relative to the bundle. The start method injects three critical dependencies: captureView for touch injection, screenRecorder for screenshots, and control for HID/clipboard operations.

Server Implementation Details

The VPhoneHostControl class in sources/vphone-cli/VPhoneHostControl.swift implements a standard Berkeley socket server using AF_UNIX. The implementation follows this sequence:

• Socket setup: Creates a SOCK_STREAM socket, binds to the filesystem path, and begins listening.
• Accept loop: The acceptLoop (lines 20‑26) runs indefinitely on a background dispatch queue, accepting client connections via accept().
• Client handling: Each connection spawns handleClient (lines 28‑122), which reads a single newline-terminated JSON command, executes the action on the main actor, and writes a JSON response before closing the connection.

The protocol mandates single-line JSON messages terminated by a newline character (\n). The server processes one command per connection, making it stateless and safe for concurrent scripted access.

JSON Command Protocol

The socket API recognizes five primary command types, distinguished by the "t" field in the JSON payload. As documented in VPhoneHostControl.swift (lines 14‑21), the supported operations are:

• "screenshot" – Captures the current display. Optionally accepts a "path" string to save a full-resolution PNG. Always returns a compact base64-encoded grayscale JPEG in the response.
• "tap" – Simulates a single touch at screen coordinates. Requires "x" and "y" integer parameters in pixels.
• "swipe" – Performs a drag gesture between two points. Requires "x1", "y1", "x2", and "y2" coordinates. Optionally accepts "ms" to specify gesture duration in milliseconds (defaults to system standard).
• "key" – Sends a HID key event. The "name" field accepts: home, power, volup, or voldown.
• "type" – Sets the guest clipboard to a specified string via the "text" field, effectively simulating text input.

Response Format

Every command receives a single-line JSON response written by writeResponse (lines 33‑45). The response object always contains:

• ok: Boolean indicating success or failure.
• path: (Optional) Filesystem path when "screenshot" includes a save location.
• error: (Optional) Error message string when ok is false.
• image: (Optional) Compact base64-encoded grayscale JPEG image, approximately 1/9th the size of the original screenshot, useful for visual verification without file I/O.

Practical Code Examples

Shell Commands with Netcat

For quick automation or shell scripts, use nc (netcat) to send JSON commands to the socket:

# Define the socket path (check console output at startup for exact location)

SOCK=/tmp/vphone.sock

# Simulate a tap at coordinate (645, 1398)

echo '{"t":"tap","x":645,"y":1398}' | nc -U $SOCK

# Perform a swipe from bottom to top over 300ms

echo '{"t":"swipe","x1":645,"y1":2600,"x2":645,"y2":1400,"ms":300}' | nc -U $SOCK

# Press the Home button

echo '{"t":"key","name":"home"}' | nc -U $SOCK

# Save a full-resolution screenshot and receive compact preview

echo '{"t":"screenshot","path":"/tmp/shot.png"}' | nc -U $SOCK

# Inject text into the guest clipboard

echo '{"t":"type","text":"Hello from the host!"}' | nc -U $SOCK

Each command outputs a JSON response line, such as:

{"ok":true,"path":"/tmp/shot.png","image":"<base64-encoded-grayscale-jpeg>"}

Python Client Implementation

For programmatic automation, Python's socket module provides a robust client:

#!/usr/bin/env python3
import socket
import json
import base64
from pathlib import Path

SOCK_PATH = "/tmp/vphone.sock"  # Adjust to match your configuration

def send_cmd(cmd: dict) -> dict:
    """Send a JSON command and return the parsed response."""
    with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as s:
        s.connect(SOCK_PATH)
        # Commands must be terminated by newline

        s.sendall(json.dumps(cmd).encode() + b'\n')
        # Read single-line response

        resp = s.recv(8192).decode().strip()
    return json.loads(resp)

# Capture compact screenshot without saving file

resp = send_cmd({"t": "screenshot"})
if resp["ok"]:
    img_data = base64.b64decode(resp["image"])
    Path("preview.jpg").write_bytes(img_data)
    print("Screenshot captured")
else:
    print("Error:", resp.get("error"))

# Tap and capture result state

resp = send_cmd({"t": "tap", "x": 645, "y": 1398})
if resp["ok"]:
    img = base64.b64decode(resp["image"])
    Path("after_tap.jpg").write_bytes(img)

Swift Native Client

For macOS applications integrating with vphone-cli, use Foundation's socket API:

import Foundation

func send(command: [String: Any], to socketPath: String) throws -> [String: Any] {
    let fd = socket(AF_UNIX, SOCK_STREAM, 0)
    defer { close(fd) }
    
    var addr = sockaddr_un()
    addr.sun_family = sa_family_t(AF_UNIX)
    let pathBytes = socketPath.utf8CString
    withUnsafeMutablePointer(to: &addr.sun_path) { ptr in
        ptr.withMemoryRebound(to: CChar.self, capacity: pathBytes.count) { dst in
            for (i, b) in pathBytes.enumerated() { dst[i] = b }
        }
    }
    
    let len = socklen_t(MemoryLayout<sockaddr_un>.size)
    try connect(fd, withUnsafePointer(to: &addr) { 
        $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { $0 } 
    }, len)
    
    let data = try JSONSerialization.data(withJSONObject: command)
    var payload = data + [0x0A] // newline
    
    try payload.withUnsafeBytes { ptr in
        var sent = 0
        while sent < payload.count {
            let n = write(fd, ptr.baseAddress! + sent, payload.count - sent)
            if n <= 0 { throw NSError(domain: "SocketError", code: -1) }
            sent += n
        }
    }
    
    var buffer = [UInt8](repeating: 0, count: 4096)
    var response = Data()
    while true {
        let n = read(fd, &buffer, buffer.count)
        if n <= 0 { break }
        response.append(contentsOf: buffer[0..<n])
        if buffer[0..<n].contains(0x0A) { break }
    }
    
    return try JSONSerialization.jsonObject(with: response) as! [String: Any]
}

// Usage example
let result = try send(command: ["t": "key", "name": "power"], to: "/tmp/vphone.sock")
print(result)

Key Implementation Files

Understanding the source architecture helps when extending functionality or debugging socket interactions:

• sources/vphone-cli/VPhoneHostControl.swift – Contains the complete socket server implementation, including acceptLoop, handleClient, and writeResponse.
• sources/vphone-cli/VPhoneAppDelegate.swift – Initializes the socket path and starts the control server during application launch.
• sources/vphone-cli/VPhoneControl.swift – Provides the underlying HID event injection and clipboard APIs used by the socket commands.
• sources/vphone-cli/VPhoneScreenRecorder.swift – Implements the private API screenshot capture that generates both full-resolution files and compact JPEG previews.
• sources/vphone-cli/VPhoneVirtualMachineView.swift – Declares the touch injection interface required for tap and swipe commands.

Summary

• vphone-cli creates a Unix-domain socket at runtime, typically located beside the VM configuration file.
• The socket accepts single-line JSON commands via TCP-like stream semantics, supporting screenshot, tap, swipe, key, and type operations.
• Responses always include an ok boolean and may contain image (base64 compact JPEG), path, or error fields.
• The architecture uses three injected services: VPhoneVirtualMachineView for input, VPhoneScreenRecorder for capture, and VPhoneControl for HID/clipboard.
• Clients can connect using standard tools like netcat, or implement native clients in Python, Swift, or any language supporting Unix sockets.

Frequently Asked Questions

Where is the vphone-cli Unix socket located?

The socket path is derived from the VM configuration file location at runtime. In VPhoneAppDelegate.swift (lines 55‑66), the code constructs the path by replacing the configuration filename with vphone.sock. If your config resides at ~/VMs/phone.cfg, expect the socket at ~/VMs/vphone.sock. Check the application console output at startup for the exact path.

What automation commands does vphone-cli support?

The socket API supports five command types: screenshot (capture display), tap (touch at x,y), swipe (drag between coordinates), key (hardware buttons like home/power/volume), and type (clipboard text injection). Each command requires a "t" field specifying the type and relevant payload parameters.

How do I capture screenshots via the socket API?

Send {"t":"screenshot"} to receive a compact base64-encoded grayscale JPEG in the image field of the response. To save a full-resolution PNG to disk, include the optional "path" parameter, such as {"t":"screenshot","path":"/tmp/capture.png"}. The response will include both the path and the compact image preview.

Can multiple clients connect to the socket simultaneously?

Yes, but each connection handles exactly one command before closing. The acceptLoop in VPhoneHostControl.swift spawns concurrent handleClient tasks on a background queue, allowing multiple scripts or applications to send commands simultaneously. However, the VM actions themselves execute serially on the main actor to prevent race conditions in the guest state.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}