Prime Agent Session JSONL Storage Format: Structure, Versioning, and Implementation

Prime Agent persists every interactive session as an append-only JSON Lines (JSONL) file where each line contains a self-contained JSON object with a mandatory type field, enabling crash-safe recovery, tree-structured branching via id and parentId relationships, and streaming-friendly processing.

The open-source Prime Agent repository by PrimeIntellect-ai uses a specialized session JSONL storage format to maintain persistent, branchable conversation histories. Each session file stores typed entries as individual JSON objects separated by newline characters (LF \n), allowing the system to reconstruct complex interaction trees while maintaining strict append-only safety guarantees. Understanding this format is essential for developers building tools that parse, migrate, or extend Prime Agent sessions.

Core Architecture of the JSONL Format

The session JSONL storage format treats each line of the file as an independent JSON object representing a single session entry. This design provides several architectural advantages for production AI agent systems.

• Append-only safety: New entries are strictly appended to the file; existing lines are never modified in-place, making crash recovery trivial and preventing data corruption during unexpected shutdowns.
• Streaming-friendly processing: Consumers read entries line-by-line with O(n) memory complexity, enabling efficient handling of multi-megabyte session files without loading the entire history into memory.
• Tree-structured branching: Each entry carries a unique id and an optional parentId, allowing the linear file format to represent complex branching conversations, merges, and rewind operations without file fragmentation.

According to the specification in packages/coding-agent/docs/session-format.md, the format uses LF (\n) as the sole delimiter, with each JSON object containing a mandatory type field that identifies the record category.

Entry Types and Schema Structure

Every line in a session file must be a valid JSON object containing at minimum a type field. The current schema (version 3) defines several core entry types that orchestrate the conversation flow.

Header entry (always first):

{ "type":"header", "version":3, "workingDir":"/home/user/project", "timestamp":1728123456789 }

Message entries:

{ "type":"message", "message": { "role":"assistant", "content":[{ "type":"text","text":"Hello!" }], "api":"openai","provider":"openai","model":"gpt-4","usage":{...},"timestamp":1728123456790 } }

Tool interaction entries:

{ "type":"toolCall", "toolCallId":"toolu_01abc","name":"read","arguments":{ "path":"README.md" },"timestamp":1728123456800 }
{ "type":"toolResult", "toolCallId":"toolu_01abc","toolName":"read","content":[{ "type":"text","text":"File contents…" }],"isError":false,"timestamp":1728123456810 }

Compaction markers:

{ "type":"compaction", "summary":"…", "tokensBefore":12345,"tokensAfter":6789,"timestamp":1728123457000 }

Additional types defined in packages/coding-agent/src/core/messages.ts include modelChange, thinkingLevel, branchSummary, and custom (renamed from hookMessage in version 3).

Session Versioning and Migration

The session JSONL storage format maintains backward compatibility through explicit versioning stored in the header entry. The implementation in packages/coding-agent/src/core/session-manager.ts handles automatic migration of legacy files on load.

Version	Description
1	Linear sequence of entries without branching support (legacy).
2	Introduced tree structure using id and parentId fields for branching.
3	Unified extensions with renamed message types; current stable version.

When Prime Agent loads a session file, it checks the version field in the header. Files using version 1 or 2 are automatically migrated to version 3 in memory, ensuring that downstream consumers always interact with the current schema while preserving historical session data.

File System Location and Session Manager

Session files are stored in the user's home directory under a predictable path structure:

~/.prime/agent/sessions/<session-id>.jsonl

The SessionManager class in packages/coding-agent/src/core/session-manager.ts encapsulates all read and write operations for these files. This module implements the append-only safety guarantees, ensuring that write operations use atomic file appends rather than read-modify-write cycles. The manager also coordinates with the type definitions in packages/ai/src/types.ts and packages/agent/src/types.ts, where the AgentMessage union type aggregates valid message payloads for the JSONL entries.

Working with Session Files Programmatically

Developers can parse and manipulate session files using standard streaming I/O. The JSONL format requires no specialized binary parsers, making it accessible from any language with JSON support.

Reading a Session File via Streaming

This TypeScript implementation processes sessions without loading the entire file into memory:

import { createReadStream } from "node:fs";
import { createInterface } from "node:readline";

async function* parseSession(filePath: string) {
  const stream = createReadStream(filePath, { encoding: "utf8" });
  const rl = createInterface({ input: stream, crlfDelay: Infinity });

  for await (const line of rl) {
    if (!line.trim()) continue;               // skip empty lines
    yield JSON.parse(line) as any;            // each line is a JSON object
  }
}

// Example usage:
for await (const entry of parseSession("/home/alice/.prime/agent/sessions/abc123.jsonl")) {
  console.log(entry.type, entry.timestamp);
}

Extracting Specific Message Types

To filter for assistant messages while maintaining type safety:

import { parseSession } from "./parse-session";

async function listAssistantMessages(sessionPath: string) {
  const msgs = [];
  for await (const entry of parseSession(sessionPath)) {
    if (entry.type === "message" && entry.message?.role === "assistant") {
      msgs.push(entry.message);
    }
  }
  return msgs;
}

Appending Entries Safely

When extending a session, always append rather than modifying existing content:

import { appendFileSync } from "node:fs";

function appendAssistantMessage(sessionPath: string, text: string) {
  const obj = {
    type: "message",
    message: {
      role: "assistant",
      content: [{ type: "text", text }],
      api: "openai",
      provider: "openai",
      model: "gpt-4",
      usage: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, totalTokens: 0, cost: {} },
      timestamp: Date.now(),
    },
  };
  appendFileSync(sessionPath, JSON.stringify(obj) + "\n", "utf8");
}

Summary

• Prime Agent stores interactive sessions as JSONL files at ~/.prime/agent/sessions/<session-id>.jsonl, with each line representing a typed JSON object.
• The format supports tree-structured branching through id and parentId fields, despite the linear file layout.
• Version 3 is the current schema standard; older versions are automatically migrated by the SessionManager in packages/coding-agent/src/core/session-manager.ts.
• Append-only writes ensure crash safety—entries are never modified in-place, only appended.
• Core entry types include header, message, toolCall, toolResult, compaction, and custom, with full type definitions available in packages/coding-agent/src/core/messages.ts.

Frequently Asked Questions

What is the difference between session version 2 and version 3?

Version 2 introduced the tree structure using id and parentId fields to support branching conversations. Version 3 unified extension mechanisms and renamed the hookMessage type to custom for better semantic clarity. Both versions support the same branching capabilities, but version 3 provides cleaner type definitions for external integrations.

How does Prime Agent handle concurrent writes to the same session file?

The SessionManager implementation relies on append-only file operations through the underlying operating system's atomic append guarantees. Since entries are never modified in-place—only appended to the end—concurrent append operations from multiple processes do not corrupt the file structure, though they may interleave entries. For strict consistency, Prime Agent typically coordinates writes through a single process.

Can I manually edit a session JSONL file without corrupting the session?

Yes, provided you maintain JSON validity per line and preserve the append-only constraint. You may insert new entries between existing lines or modify non-essential fields, but you must not alter the id or parentId relationships if you wish to maintain the conversation tree integrity. Changing historical entries (anything before the final line) is safe for read-only operations but risks breaking consistency if the session is loaded by the SessionManager expecting specific cryptographic or usage-based checksums.

What is the maximum file size supported for Prime Agent sessions?

There is no hard file size limit in the JSONL format specification itself. The streaming parser in session-manager.ts processes entries line-by-line with O(1) memory consumption relative to file size. However, extremely large files (gigabytes) may impact startup times when the system builds the conversation tree index. For long-running sessions, Prime Agent uses compaction entries to summarize and truncate older conversation branches, effectively capping the file growth while preserving context.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}