Pi Web .jsonl Session File Format: Complete Structure and Parsing Guide

Pi Web uses a line-delimited JSON (.jsonl) format where each line represents a distinct session event—headers, messages, model changes, and compaction summaries—parsed by helper functions in lib/session-reader.ts.

The .jsonl session file format is central to how Pi Web stores and replays conversational AI sessions. Every chat session is persisted as a single .jsonl file on disk, with each line containing a self-contained JSON object. This design enables efficient appending, random access to the session header, and streaming parsing. This guide breaks down the exact structure of these files and how pi-web interprets them.

Core Structure of a .jsonl Session File

A .jsonl session file contains multiple entry types, each identified by a type field. The file must begin with a session header entry. Subsequent lines can appear in any order, though they typically follow chronological sequence.

Entry Type	Key Fields	Purpose
session	type, version, id, timestamp, cwd, parentSession	First line; identifies the file and session metadata.
model_change	type, provider, modelId	Records when the LLM provider or model changes.
message	type, message (with role: user, assistant, toolResult, bashExecution)	Core chat content including tool interactions.
toolResult	toolCallId, content	Embedded within message entries; returns tool execution results.
compaction	type, summary, firstKeptEntryId, tokensBefore	Summarizes pruned conversation history to manage context window.
session_info	type, name	Optional UI metadata such as user-defined session names.

The complete schema is documented in [AGENTS.md](https://github.com/agegr/pi-web/blob/main/AGENTS.md#pi-session-file-format) in the pi-web repository.

Example .jsonl File Contents

{"type":"session","version":3,"id":"c5f0a1b2","timestamp":"2023-04-01T12:00:00Z","cwd":"/home/user/project"}
{"type":"model_change","id":"01ab23cd","provider":"openai","modelId":"gpt-4"}
{"type":"message","id":"02cd45ef","parentId":null,"message":{"role":"user","content":"Explain the .jsonl format"}}
{"type":"message","id":"03ef67ab","parentId":"02cd45ef","message":{"role":"assistant","content":[{"type":"text","text":"The .jsonl format uses one JSON object per line..."}]}}
{"type":"message","id":"04ab89cd","parentId":"03ef67ab","message":{"role":"assistant","content":[],"tool_calls":[{"type":"toolCall","id":"call_abc123","name":"read_file","arguments":{"path":"/docs/spec.md"}}]}}
{"type":"message","id":"05cd01ef","parentId":"04ab89cd","message":{"role":"toolResult","toolCallId":"call_abc123","content":"File contents here..."}}

How pi-web Parses .jsonl Session Files

The pi-web codebase implements a layered parsing pipeline in lib/session-reader.ts. Each layer handles a specific concern: file location, header validation, entry extraction, and UI transformation.

1. Locating Session Files

The listAllSessions() function in lib/session-reader.ts (lines 17-46) builds a mapping of session IDs to file paths:

// lib/session-reader.ts#L17-L46
import { SessionManager } from '@agegr/pi-sdk';

export async function listAllSessions(): Promise<Map<string, string>> {
  const sessions = await SessionManager.listAll();
  const map = new Map<string, string>();
  for (const session of sessions) {
    map.set(session.id, session.path);
  }
  return map;
}

This uses the Pi SDK's SessionManager.listAll() and caches results for performance.

2. Reading and Validating the Session Header

The readSessionHeader(filePath) function (lines 66-94) performs a fast header check by reading only the first 64KB of the file:

// lib/session-reader.ts#L66-L94
import { open } from 'fs/promises';

export async function readSessionHeader(filePath: string): Promise<SessionHeader | null> {
  const fd = await open(filePath, 'r');
  try {
    const buffer = Buffer.alloc(65536); // 64KB max header read
    const { bytesRead } = await fd.read(buffer, 0, 65536, 0);
    const content = buffer.toString('utf8', 0, bytesRead);
    const firstNewline = content.indexOf('\n');
    if (firstNewline === -1) return null;
    
    const firstLine = content.slice(0, firstNewline);
    const parsed = JSON.parse(firstLine);
    return parsed.type === 'session' ? parsed : null;
  } finally {
    await fd.close();
  }
}

This validates that the file is a legitimate Pi session before loading the entire contents.

3. Loading All Session Entries

For full session reconstruction, getSessionEntries(filePath) (lines 99-102) delegates to the SDK's streaming reader:

// lib/session-reader.ts#L99-L102
export function getSessionEntries(filePath: string): SessionEntry[] {
  const session = SessionManager.open(filePath);
  return session.getEntries();
}

The SDK handles the line-by-line JSON parsing and returns a typed SessionEntry[] array.

4. Building UI-Ready Context

The buildSessionContext(entries, leafId, options) function (lines 104-140) transforms raw entries into React-friendly structures:

• Creates a byId map for O(1) entry lookup
• Calls piBuildSessionContext and piBuildContextEntries SDK helpers for branch selection
• Converts each entry via entryToUiMessage

// Simplified usage pattern from lib/session-reader.ts
export function buildSessionContext(
  entries: SessionEntry[],
  leafId?: string,
  options?: BuildOptions
): SessionContext {
  const byId = new Map(entries.map(e => [e.id, e]));
  const { selectedIds } = piBuildSessionContext(entries, leafId, options);
  const messages = selectedIds.map(id => entryToUiMessage(byId.get(id)!, options));
  return { messages, entryIds: selectedIds, byId };
}

5. Entry-to-Message Conversion

The entryToUiMessage(entry, options) function handles type-specific transformations:

Entry Type	Conversion Behavior
message	Normalizes tool calls via normalizeToolCalls; optionally strips base64 images; defers "thinking" blocks
compaction	Creates a custom UI message with summary text and token statistics
branch_summary	Converts to user-visible message with branch metadata
custom_message	Preserves custom payload as message content

Tool call normalization occurs in lib/normalize.ts through the normalizeToolCalls function, which reconciles SDK shape {type:"toolCall", id, name, arguments} with UI shape {toolCallId, toolName, input}.

Complete Parsing Pipeline

listAllSessions() ──► resolveSessionPath(sessionId)
                          │
                          ▼
              readSessionHeader(path) ──► validates "session" type
                          │
                          ▼
              getSessionEntries(path) ──► SessionEntry[]
                          │
                          ▼
              buildSessionContext() ──► entryToUiMessage() per entry
                          │
                          ▼
                    { messages: AgentMessage[], entryIds: string[] }

Working with .jsonl Files in Code

Load a Complete Session Context

import {
  resolveSessionPath,
  readSessionHeader,
  getSessionEntries,
  buildSessionContext
} from '@/lib/session-reader';

async function loadSessionContext(sessionId: string) {
  const path = await resolveSessionPath(sessionId);
  if (!path) throw new Error('Session not found');

  const header = await readSessionHeader(path);
  if (!header) throw new Error('Invalid session file: missing header');

  const entries = getSessionEntries(path);
  const ctx = buildSessionContext(entries);
  
  return ctx; // { messages: AgentMessage[], entryIds: string[], byId: Map }
}

Type Definitions Reference

Key types are defined in lib/pi-types.ts:

• SessionHeader — the mandatory first-line session object
• SessionEntry — union type of all possible line entries
• AgentMessage — UI-facing message format after transformation

Key Implementation Files

File	Responsibility
lib/session-reader.ts	Core parsing logic: listAllSessions, readSessionHeader, getSessionEntries, buildSessionContext, entryToUiMessage
lib/normalize.ts	normalizeToolCalls — SDK-to-UI tool call shape conversion
lib/pi-types.ts	TypeScript interfaces for SessionHeader, SessionEntry, AgentMessage
AGENTS.md	Official .jsonl schema documentation

Summary

• .jsonl format: One JSON object per line, starting with mandatory session header
• Entry types: session, model_change, message, compaction, session_info, plus embedded toolResult
• Parsing layers: File location → header validation → entry extraction → UI transformation
• Core functions: listAllSessions(), readSessionHeader(), getSessionEntries(), buildSessionContext()
• Tool handling: normalizeToolCalls() reconciles SDK and UI payload shapes
• Performance: 64KB header read for validation; full read only when needed

Frequently Asked Questions

What makes .jsonl better than a single JSON file for Pi sessions?

Line-delimited JSON allows append-only writes for new messages without rewriting the entire file, supports streaming parsing for large sessions, and enables fast header validation by reading just the first line. The pi-web implementation leverages all three advantages through readSessionHeader()'s 64KB limited read and getSessionEntries()'s SDK-backed streaming.

How does pi-web handle branched conversations in .jsonl files?

The buildSessionContext() function uses SDK helpers piBuildSessionContext and piBuildContextEntries to compute the correct entry sequence for a given leafId. The parentId fields in message entries form a tree, and the algorithm selects the path from root to the specified leaf, enabling the branch-aware UI.

What happens if a .jsonl file is missing the session header?

readSessionHeader() returns null if the first line's type field is not "session". This causes the loading pipeline to throw an error, preventing corrupted or non-session files from being processed as valid conversations.

Are tool results stored inline or referenced externally?

Tool results are embedded inline as message entries with role: "toolResult" containing toolCallId and content fields. The entryToUiMessage() function extracts these and correlates them with their originating tool calls through the toolCallId identifier.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}