# What Kind of Data Does PrimeAgent Handle? A Complete Data Architecture Guide

> Discover what kind of data PrimeAgent handles including text, JSON, images, tool calls, and more. Explore its comprehensive data architecture for seamless AI agent communication.

- Repository: [Prime Intellect/prime-agent](https://github.com/PrimeIntellect-ai/prime-agent)
- Tags: data-architecture-guide
- Published: 2026-09-08

---

**PrimeAgent handles plain text, structured JSON messages, images, tool calls, session transcripts, artifacts, skill metadata, and sub-agent messages—all as typed, JSON-serializable data flowing through a daemon-backed pipeline.**

PrimeAgent is an open-source coding agent framework developed by PrimeIntellect. Understanding what data types flow through its architecture is essential for building custom integrations, debugging session behavior, or extending the system with new capabilities. This guide breaks down every data category PrimeAgent processes, how each is represented, and where it travels in the codebase.

---

## Core Data Types in PrimeAgent

PrimeAgent's architecture separates UI (client), execution (worker), and persistence (storage) into distinct layers. Every data type crossing these boundaries is strictly typed and logged for deterministic replay. Below are the eight primary categories.

---

### Plain Text Prompts and Responses

The simplest data form: strings exchanged between users, models, and the UI.

Text appears in the `message` field of prompt objects or as `text` events from model providers. The typical flow spans multiple components:

1. **User** enters text via CLI/TUI or JSON/RPC client
2. **AgentConnection** receives and routes the message
3. **Supervisor** coordinates worker assignment
4. **Worker** forwards to **AgentSession**
5. **Model provider** generates a response
6. Return path reverses through the same components

In [`packages/coding-agent/docs/architecture.md`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/packages/coding-agent/docs/architecture.md), this pipeline is visualized as a clean separation where no component directly touches another's internal state.

---

### Structured Host Requests and Host Replies

The TypeScript-Python boundary uses a strict request-reply protocol defined in [`prime-agent-runtime/src/rlm/repl.md`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/prime-agent-runtime/src/rlm/repl.md).

Each exchange carries:
- `type: "host_request"` or `type: "host_reply"`
- An `id` field that ties request to reply
- A `data` payload with operation-specific content

```ts
// Example: Requesting a file read from Python kernel
await rlm.repl.host_request({
  type: "host_request",
  id: "req-42",
  data: { op: "readFile", path: "./data.csv" }
});

```

The Python kernel processes the operation and returns a `host_reply` with matching `id`. This design enables async, out-of-order responses while maintaining causal linking.

---

### Image Content

PrimeAgent supports multimodal prompts where users attach images for analysis.

Images are represented as `ImageContent` objects defined in [`scripts/tool-stats.ts`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/scripts/tool-stats.ts):

```ts
{
  type: "image",
  data: "iVBORw0KGgoAAAANSUhEUgAA...", // base64-encoded
  mimeType: "image/png"
}

```

The full flow:
- User attaches image to prompt
- `AgentConnection` serializes and transmits
- Model provider receives and processes
- Generated description or analysis returns as text
- UI renders via [`packages/tui/src/terminal-image.ts`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/packages/tui/src/terminal-image.ts) (terminal graphics or metadata fallback)

---

### Tool Calls and Tool Results

When the model decides to execute an action—run bash, read a file, search the web—it emits a `ToolCallContent` object.

Structure from [`scripts/tool-stats.ts`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/scripts/tool-stats.ts):
- `type: "toolCall"`
- `id`: unique identifier
- `name`: tool to invoke
- `arguments`: parameters as key-value map

```ts
function handleToolCall(call: ToolCallContent) {
  if (call.name === "bash") {
    const result = execSync(call.arguments?.cmd ?? "", { encoding: "utf-8" });
    sendHostReply(call.id, { stdout: result });
  }
}

```

The `AgentSession` routes tool calls to either built-in handlers or the Python kernel, then packages results back to the model as `host_reply` data.

---

### Session Transcript Events

Every interaction is persisted as `JsonEvent` objects for debugging, replay, and recovery.

Defined in [`scripts/session-transcripts.ts`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/scripts/session-transcripts.ts), events contain:
- `type`: message category (user, assistant, system, tool)
- `id`: unique identifier
- `timestamp`: ISO 8601 string
- `data`: event payload
- `details`: optional metadata (files read, tools used)

```ts
import { appendEntry } from "./scripts/session-transcripts";

await appendEntry({
  type: "assistant",
  id: "msg-99",
  timestamp: new Date().toISOString(),
  data: { text: "Here is the summary of your data." },
  details: { readFiles: ["./data.csv"] }
});

```

Transcripts write as JSONL after each turn. The supervisor reads them during session attach or crash recovery.

---

### Artifacts and Supplemental State

Files, images, skill packages, and binary blobs generated during execution become **artifacts**.

Storage details from [`packages/coding-agent/docs/architecture.md`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/packages/coding-agent/docs/architecture.md):
- Written to session-scoped storage directory
- Referenced by JSON entry with `artifactId`, `path`, `mimeType`
- Reloaded on session attach or resume

Artifacts decouple large binary data from the transcript stream, keeping logs compact while preserving full session state.

---

### Skill Metadata

Skills—reusable capabilities defined in markdown and Python—carry metadata describing inputs, outputs, and configuration.

Example from [`packages/coding-agent/skills/websearch/SKILL.md`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/packages/coding-agent/skills/websearch/SKILL.md):
- Arbitrary key-value `metadata` maps
- Optional asset file references
- Activation via `/skill:<name>` command

The metadata informs the agent how to invoke the skill and interpret its results, making skills self-documenting composable units.

---

### Sub-Agent Messages

PrimeAgent supports parallel or background work through child RLM runtimes.

Parent-child communication uses the same JSON envelope as host requests/replies, but routed through a "children" subtree in the runtime graph per [`packages/coding-agent/docs/architecture.md`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/packages/coding-agent/docs/architecture.md):

```

Parent AgentSession → child RLM runtime → provider → child reply → parent

```

This enables hierarchical agent architectures where a supervisor delegates to specialized sub-agents.

---

## Data Flow Summary

| Data Category | Key File | Representation |
|-------------|----------|---------------|
| Plain text | [`architecture.md`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/architecture.md) | String in `message` field |
| Host requests/replies | [`repl.md`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/repl.md) | JSON with `type`, `id`, `data` |
| Images | [`tool-stats.ts`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/tool-stats.ts) | `ImageContent` object |
| Tool calls | [`tool-stats.ts`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/tool-stats.ts) | `ToolCallContent` object |
| Transcripts | [`session-transcripts.ts`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/session-transcripts.ts) | `JsonEvent` with full metadata |
| Artifacts | [`architecture.md`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/architecture.md) | Storage reference + JSON descriptor |
| Skill metadata | [`SKILL.md`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/SKILL.md) files | Key-value maps |
| Sub-agent messages | [`architecture.md`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/architecture.md) | Routed JSON envelopes |

---

## Key Source Files for Data Types

- **[`packages/coding-agent/docs/architecture.md`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/packages/coding-agent/docs/architecture.md)** — High-level data flow diagram
- **[`prime-agent-runtime/src/rlm/repl.md`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/prime-agent-runtime/src/rlm/repl.md)** — Host request/reply protocol specification
- **[`scripts/tool-stats.ts`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/scripts/tool-stats.ts)** — Type definitions for `ImageContent` and `ToolCallContent`
- **[`scripts/session-transcripts.ts`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/scripts/session-transcripts.ts)** — `JsonEvent` structure and persistence helpers
- **[`packages/coding-agent/skills/websearch/SKILL.md`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/packages/coding-agent/skills/websearch/SKILL.md)** — Skill metadata example
- **[`packages/tui/src/terminal-image.ts`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/packages/tui/src/terminal-image.ts)** — Image rendering implementation

---

## Summary

PrimeAgent handles eight distinct data categories, each with explicit typing and serialization:

- **Plain text** flows through the full UI-to-model pipeline
- **Structured JSON** enables TypeScript-Python interoperation via host requests/replies
- **Images** travel as base64-encoded `ImageContent` objects
- **Tool calls** trigger execution with typed arguments and results
- **Transcript events** provide complete session logging and recovery
- **Artifacts** store binary state outside the log stream
- **Skill metadata** makes capabilities self-describing
- **Sub-agent messages** support hierarchical runtime composition

All data is **JSON-serializable**, **persisted**, and **replayable**—core guarantees that make PrimeAgent deterministic and debuggable.

---

## Frequently Asked Questions

### Does PrimeAgent support multimodal inputs like images?

Yes. PrimeAgent accepts `ImageContent` objects containing base64-encoded image data with MIME type annotations. These pass through `AgentConnection` to model providers that support vision capabilities. The TUI renders images via terminal graphics or falls back to compact metadata display.

### How does PrimeAgent persist session data for later recovery?

Session data persists as JSONL transcript files using `JsonEvent` objects from [`scripts/session-transcripts.ts`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/scripts/session-transcripts.ts). Each event includes a timestamp, unique ID, data payload, and optional details. The supervisor reads these transcripts when reattaching to a session or recovering from crashes.

### What protocol connects the TypeScript frontend to the Python runtime?

The **RLM REPL protocol** defined in [`prime-agent-runtime/src/rlm/repl.md`](https://github.com/PrimeIntellect-ai/prime-agent/blob/main/prime-agent-runtime/src/rlm/repl.md). It uses `host_request` and `host_reply` JSON envelopes with matching `id` fields for asynchronous, ordered communication between the TypeScript `AgentSession` and Python kernel.

### Can PrimeAgent spawn child agents for parallel tasks?

Yes. PrimeAgent supports sub-agent runtimes that communicate via the same JSON envelope format as host requests. Messages route through a "children" subtree in the runtime graph, enabling parent agents to delegate work and aggregate results from multiple concurrent child sessions.