What Kind of Data Does PrimeAgent Handle? A Complete Data Architecture Guide

PrimeAgent handles plain text, structured JSON messages, images, tool calls, session transcripts, artifacts, skill metadata, and sub-agent messages—all as typed, JSON-serializable data flowing through a daemon-backed pipeline.

PrimeAgent is an open-source coding agent framework developed by PrimeIntellect. Understanding what data types flow through its architecture is essential for building custom integrations, debugging session behavior, or extending the system with new capabilities. This guide breaks down every data category PrimeAgent processes, how each is represented, and where it travels in the codebase.

---

Core Data Types in PrimeAgent

PrimeAgent's architecture separates UI (client), execution (worker), and persistence (storage) into distinct layers. Every data type crossing these boundaries is strictly typed and logged for deterministic replay. Below are the eight primary categories.

---

Plain Text Prompts and Responses

The simplest data form: strings exchanged between users, models, and the UI.

Text appears in the message field of prompt objects or as text events from model providers. The typical flow spans multiple components:

1. User enters text via CLI/TUI or JSON/RPC client
2. AgentConnection receives and routes the message
3. Supervisor coordinates worker assignment
4. Worker forwards to AgentSession
5. Model provider generates a response
6. Return path reverses through the same components

In packages/coding-agent/docs/architecture.md, this pipeline is visualized as a clean separation where no component directly touches another's internal state.

---

Structured Host Requests and Host Replies

The TypeScript-Python boundary uses a strict request-reply protocol defined in prime-agent-runtime/src/rlm/repl.md.

Each exchange carries:

• type: "host_request" or type: "host_reply"
• An id field that ties request to reply
• A data payload with operation-specific content

// Example: Requesting a file read from Python kernel
await rlm.repl.host_request({
  type: "host_request",
  id: "req-42",
  data: { op: "readFile", path: "./data.csv" }
});

The Python kernel processes the operation and returns a host_reply with matching id. This design enables async, out-of-order responses while maintaining causal linking.

---

Image Content

PrimeAgent supports multimodal prompts where users attach images for analysis.

Images are represented as ImageContent objects defined in scripts/tool-stats.ts:

{
  type: "image",
  data: "iVBORw0KGgoAAAANSUhEUgAA...", // base64-encoded
  mimeType: "image/png"
}

The full flow:

• User attaches image to prompt
• AgentConnection serializes and transmits
• Model provider receives and processes
• Generated description or analysis returns as text
• UI renders via packages/tui/src/terminal-image.ts (terminal graphics or metadata fallback)

---

Tool Calls and Tool Results

When the model decides to execute an action—run bash, read a file, search the web—it emits a ToolCallContent object.

Structure from scripts/tool-stats.ts:

• type: "toolCall"
• id: unique identifier
• name: tool to invoke
• arguments: parameters as key-value map

function handleToolCall(call: ToolCallContent) {
  if (call.name === "bash") {
    const result = execSync(call.arguments?.cmd ?? "", { encoding: "utf-8" });
    sendHostReply(call.id, { stdout: result });
  }
}

The AgentSession routes tool calls to either built-in handlers or the Python kernel, then packages results back to the model as host_reply data.

---

Session Transcript Events

Every interaction is persisted as JsonEvent objects for debugging, replay, and recovery.

Defined in scripts/session-transcripts.ts, events contain:

• type: message category (user, assistant, system, tool)
• id: unique identifier
• timestamp: ISO 8601 string
• data: event payload
• details: optional metadata (files read, tools used)

import { appendEntry } from "./scripts/session-transcripts";

await appendEntry({
  type: "assistant",
  id: "msg-99",
  timestamp: new Date().toISOString(),
  data: { text: "Here is the summary of your data." },
  details: { readFiles: ["./data.csv"] }
});

Transcripts write as JSONL after each turn. The supervisor reads them during session attach or crash recovery.

---

Artifacts and Supplemental State

Files, images, skill packages, and binary blobs generated during execution become artifacts.

Storage details from packages/coding-agent/docs/architecture.md:

• Written to session-scoped storage directory
• Referenced by JSON entry with artifactId, path, mimeType
• Reloaded on session attach or resume

Artifacts decouple large binary data from the transcript stream, keeping logs compact while preserving full session state.

---

Skill Metadata

Skills—reusable capabilities defined in markdown and Python—carry metadata describing inputs, outputs, and configuration.

Example from packages/coding-agent/skills/websearch/SKILL.md:

• Arbitrary key-value metadata maps
• Optional asset file references
• Activation via /skill:<name> command

The metadata informs the agent how to invoke the skill and interpret its results, making skills self-documenting composable units.

---

Sub-Agent Messages

PrimeAgent supports parallel or background work through child RLM runtimes.

Parent-child communication uses the same JSON envelope as host requests/replies, but routed through a "children" subtree in the runtime graph per packages/coding-agent/docs/architecture.md:

Parent AgentSession → child RLM runtime → provider → child reply → parent

This enables hierarchical agent architectures where a supervisor delegates to specialized sub-agents.

---

Data Flow Summary

Data Category	Key File	Representation
Plain text	architecture.md	String in message field
Host requests/replies	repl.md	JSON with type, id, data
Images	tool-stats.ts	ImageContent object
Tool calls	tool-stats.ts	ToolCallContent object
Transcripts	session-transcripts.ts	JsonEvent with full metadata
Artifacts	architecture.md	Storage reference + JSON descriptor
Skill metadata	SKILL.md files	Key-value maps
Sub-agent messages	architecture.md	Routed JSON envelopes

---

Key Source Files for Data Types

• packages/coding-agent/docs/architecture.md — High-level data flow diagram
• prime-agent-runtime/src/rlm/repl.md — Host request/reply protocol specification
• scripts/tool-stats.ts — Type definitions for ImageContent and ToolCallContent
• scripts/session-transcripts.ts — JsonEvent structure and persistence helpers
• packages/coding-agent/skills/websearch/SKILL.md — Skill metadata example
• packages/tui/src/terminal-image.ts — Image rendering implementation

---

Summary

PrimeAgent handles eight distinct data categories, each with explicit typing and serialization:

• Plain text flows through the full UI-to-model pipeline
• Structured JSON enables TypeScript-Python interoperation via host requests/replies
• Images travel as base64-encoded ImageContent objects
• Tool calls trigger execution with typed arguments and results
• Transcript events provide complete session logging and recovery
• Artifacts store binary state outside the log stream
• Skill metadata makes capabilities self-describing
• Sub-agent messages support hierarchical runtime composition

All data is JSON-serializable, persisted, and replayable—core guarantees that make PrimeAgent deterministic and debuggable.

---

Frequently Asked Questions

Does PrimeAgent support multimodal inputs like images?

Yes. PrimeAgent accepts ImageContent objects containing base64-encoded image data with MIME type annotations. These pass through AgentConnection to model providers that support vision capabilities. The TUI renders images via terminal graphics or falls back to compact metadata display.

How does PrimeAgent persist session data for later recovery?

Session data persists as JSONL transcript files using JsonEvent objects from scripts/session-transcripts.ts. Each event includes a timestamp, unique ID, data payload, and optional details. The supervisor reads these transcripts when reattaching to a session or recovering from crashes.

What protocol connects the TypeScript frontend to the Python runtime?

The RLM REPL protocol defined in prime-agent-runtime/src/rlm/repl.md. It uses host_request and host_reply JSON envelopes with matching id fields for asynchronous, ordered communication between the TypeScript AgentSession and Python kernel.

Can PrimeAgent spawn child agents for parallel tasks?

Yes. PrimeAgent supports sub-agent runtimes that communicate via the same JSON envelope format as host requests. Messages route through a "children" subtree in the runtime graph, enabling parent agents to delegate work and aggregate results from multiple concurrent child sessions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}