Using Earendil Pi in RPC Mode: JSON-L Protocol Integration Guide

Earendil π provides a headless JSON-L protocol that enables programmatic control via stdin/stdout, allowing external programs to drive the coding agent without a terminal UI by sending structured commands to the interfaces defined in packages/coding-agent/src/modes/rpc/rpc-types.ts.

The earendil-works/pi repository exposes a powerful RPC mode that removes the interactive terminal interface in favor of a machine-readable stream. This mode is ideal for embedding the coding agent into IDEs, web backends, or automation scripts that require fine-grained control over the agent lifecycle. By launching the CLI with --mode rpc, you establish a bidirectional JSON Lines (JSON-L) communication channel where your application issues commands and consumes real-time events.

Protocol Architecture and Framing

The RPC implementation centers on strict JSON-L formatting rules defined in packages/coding-agent/docs/rpc.md and enforced by the TypeScript types in packages/coding-agent/src/modes/rpc/rpc-types.ts. The architecture spans three layers: the CLI entry point in packages/coding-agent/src/index.ts, the RPC engine in packages/coding-agent/src/modes/rpc/rpc-client.ts, and the agent core in packages/agent/src/core/agent-session.ts.

JSON-L Framing Requirements

All messages use LF (\n) only as a delimiter. The implementation deliberately avoids the Node.js readline module because it incorrectly splits on Unicode line separators (U+2028, U+2029) that are legal inside JSON strings. Clients must strip potential trailing \r characters before parsing.

• Commands: One JSON object per line written to stdin
• Responses: JSON objects with "type":"response" confirming command success or failure
• Events: Asynchronous JSON objects streamed to stdout during agent processing

Message Correlation

Commands may include an optional "id" field; the corresponding response echoes this identifier for correlation. Events contain a "type" field but no id, as they represent unsolicited notifications rather than request/response pairs.

Core RPC Commands

The RpcCommand union type in packages/coding-agent/src/modes/rpc/rpc-types.ts defines the instruction set for controlling the agent.

Session and Execution Control

• prompt: Sends a user message (optionally with images) to the agent
• steer: Queues a steering message while the agent is streaming a response
• follow_up: Queues a message to execute after the current turn finishes
• abort: Stops the current operation immediately
• new_session: Starts a fresh session with optional parent tracking
• bash: Executes a shell command; results appear on the next prompt cycle

Configuration and Introspection

• set_model, cycle_model, get_available_models: Control LLM selection via the interfaces defined in packages/ai/src/types.ts
• set_thinking_level, cycle_thinking_level: Adjust reasoning depth
• compact, set_auto_compaction: Manage context window size in packages/agent/src/core/agent-session.ts
• set_auto_retry, abort_retry: Configure automatic retry on transient API errors
• get_state, get_messages, get_commands: Query internal agent state

Event Stream Processing

The agent emits a rich event stream defined in packages/coding-agent/docs/rpc.md. Your client must parse these asynchronous messages to track agent activity and render streaming output.

Primary Event Types

• message_update: Streaming deltas including text_delta, thinking_delta, and toolcall_* fragments
• tool_execution_start, tool_execution_update, tool_execution_end: Progress tracking for tool invocations like bash
• queue_update: Notifications about changes to the steering and follow-up queues
• compaction_start, compaction_end: Context reduction lifecycle events
• auto_retry_start, auto_retry_end: Automatic retry status after rate-limit or overload errors

Agent Lifecycle Events

The agent_end event signals completion of the current turn, allowing your client to reset streaming buffers or prompt for new input.

Extension UI Sub-Protocol

When extensions request UI interactions via ctx.ui.select() or ctx.ui.confirm(), the RPC mode converts these into extension_ui_request events. Your client must reply with a matching extension_ui_response containing the user's input.

Supported UI Methods

The RpcExtensionUIRequest type in packages/coding-agent/src/modes/rpc/rpc-types.ts supports:

• select: Returns the chosen option string or {cancelled:true}
• confirm: Returns {confirmed:true} or {confirmed:false}
• input: Returns {value:"user text"}
• editor: Returns multi-line edited text

Optional timeout fields cause the agent to auto-resolve with undefined (or false for confirmations) if the client does not respond in time.

Client Implementation Examples

Python Raw Subprocess Client

Use standard subprocess to manage the JSON-L stream. The key is maintaining separate threads or iterators for stdin writes and stdout reads.

import subprocess, json

proc = subprocess.Popen(
    ["pi", "--mode", "rpc", "--no-session"],
    stdin=subprocess.PIPE, stdout=subprocess.PIPE, text=True
)

def send(cmd):
    proc.stdin.write(json.dumps(cmd) + "\n")
    proc.stdin.flush()

def read_events():
    for line in proc.stdout:
        yield json.loads(line)

# Send a prompt

send({"type": "prompt", "message": "List the files in the current directory."})

# Consume events

for ev in read_events():
    if ev.get("type") == "message_update":
        delta = ev.get("assistantMessageEvent", {})
        if delta.get("type") == "text_delta":
            print(delta["delta"], end="", flush=True)
    if ev.get("type") == "agent_end":
        print()
        break

Node.js Raw Implementation

When implementing in Node.js, manually handle the byte buffer to respect the LF-only framing rules without using the readline module.

const { spawn } = require("child_process");
const { StringDecoder } = require("string_decoder");

const pi = spawn("pi", ["--mode", "rpc", "--no-session"]);

function jsonlReader(stream, onLine) {
  const decoder = new StringDecoder("utf8");
  let buf = "";
  stream.on("data", (chunk) => {
    buf += typeof chunk === "string" ? chunk : decoder.write(chunk);
    while (true) {
      const i = buf.indexOf("\n");
      if (i === -1) break;
      let line = buf.slice(0, i);
      buf = buf.slice(i + 1);
      if (line.endsWith("\r")) line = line.slice(0, -1);
      onLine(line);
    }
  });
  stream.on("end", () => {
    buf += decoder.end();
    if (buf) onLine(buf.endsWith("\r") ? buf.slice(0, -1) : buf);
  });
}

jsonlReader(pi.stdout, (line) => {
  const ev = JSON.parse(line);
  if (ev.type === "message_update") {
    const d = ev.assistantMessageEvent;
    if (d && d.type === "text_delta") process.stdout.write(d.delta);
  }
  if (ev.type === "agent_end") console.log();
});

pi.stdin.write(JSON.stringify({ type: "prompt", message: "Explain RPC mode." }) + "\n");

process.on("SIGINT", () => {
  pi.stdin.write(JSON.stringify({ type: "abort" }) + "\n");
});

TypeScript RpcClient Wrapper

For TypeScript projects, import the official RpcClient from packages/coding-agent/src/modes/rpc/rpc-client.ts to avoid manual stream handling.

import { RpcClient } from "@earendil-works/pi-coding-agent";

async function demo() {
  const client = new RpcClient(["--mode", "rpc", "--no-session"]);
  await client.start();

  const reply = await client.prompt("Summarize the repository structure.");
  console.log("Assistant:", reply);

  await client.shutdown();
}
demo();

Key Source Files

Understanding the implementation requires reference to these specific files in the earendil-works/pi repository:

• packages/coding-agent/docs/rpc.md: Human-readable protocol specification, command list, and streaming rules
• packages/coding-agent/src/modes/rpc/rpc-types.ts: TypeScript definitions for RpcCommand, RpcResponse, and RpcExtensionUIRequest
• packages/coding-agent/src/modes/rpc/rpc-client.ts: High-level RpcClient class used by library consumers
• packages/agent/src/core/agent-session.ts: Core session management and model call orchestration
• packages/ai/src/types.ts: Model description and message shapes (UserMessage, AssistantMessage, ToolResultMessage)
• packages/coding-agent/src/index.ts: CLI entry point that parses --mode flags and launches the RPC engine
• packages/coding-agent/examples/extensions/rpc-demo.ts: Example extension demonstrating UI request/response flows

Summary

• Earendil π RPC mode exposes a headless JSON-L interface via stdin/stdout, defined in packages/coding-agent/src/modes/rpc/rpc-types.ts
• Strict framing requires LF-only delimiters without using standard readline parsers that mishandle Unicode separators
• Command types include prompt, steer, bash, and configuration methods like set_model and compact
• Event streaming delivers real-time updates through message_update, tool_execution_*, and lifecycle events like agent_end
• UI interactions require bidirectional handling of extension_ui_request and extension_ui_response pairs
• Client options range from raw subprocess management (Python/Node.js) to the typed RpcClient wrapper for TypeScript

Frequently Asked Questions

How do I start Earendil π in RPC mode?

Launch the binary with the --mode rpc flag. For stateless integration, add --no-session to prevent persistence. According to packages/coding-agent/src/index.ts, this flag routes execution to the RPC engine instead of the interactive terminal UI.

What is the difference between commands and events in the protocol?

Commands are request/response pairs sent to stdin that trigger actions and return confirmation via type: "response" messages. Events are asynchronous notifications streamed to stdout (like message_update or tool_execution_start) that contain no correlation ID because they are not responses to specific requests.

How do I handle interactive UI prompts when using RPC mode?

When extensions call ctx.ui.select() or ctx.ui.confirm(), the agent emits an extension_ui_request event on stdout. Your client must capture this event and reply with an extension_ui_response command on stdin containing the user's selection or cancellation status, as defined in the RpcExtensionUIRequest type.

Can I use the official client library instead of raw subprocess management?

Yes. The repository provides the RpcClient class in packages/coding-agent/src/modes/rpc/rpc-client.ts, which handles JSON-L framing, command serialization, and event parsing internally. This allows you to use high-level async methods like client.prompt() instead of managing raw byte streams.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}