Understanding the Pi Web .jsonl Session File Format and Entry Normalization

The Pi Web .jsonl session format is a line-delimited JSON structure where each entry has a type field, and Pi Web normalizes SDK entries to UI types through parallel entryIds[] and messages[] arrays with tool-call field mapping in normalize.ts.

Pi Web stores every chat session as a .jsonl (JSON Lines) file, converting raw Pi SDK entries into UI-ready messages through a careful normalization process. This article explains the complete file format and the transformation pipeline that bridges storage and display.

---

What Is the .jsonl Session File Format?

Pi Web persists sessions as line-delimited JSON files in the user's Pi agent directory, typically under ~/.pi/agent/sessions/. Each line represents a distinct entry with a mandatory "type" field that determines how Pi Web processes it.

Entry Types and Structure

Line Position	type	Purpose
First	session	Header with version, ID, timestamp, working directory, and parent session reference for forks
Variable	model_change	Records provider/model switches (e.g., "zenmux" → "claude-sonnet-4-6")
Majority	message	Chat messages: user, assistant, tool results, or bash executions
Occasional	compaction	Summarizes pruned history with summary, firstKeptEntryId, tokensBefore
Optional	session_info	User-defined metadata like session name

Example Session File

{"type":"session","version":3,"id":"<uuid>","timestamp":"2024-01-15T09:30:00Z","cwd":"/home/user/project","parentSession":null}
{"type":"model_change","id":"a1b2c3d4","parentId":null,"provider":"zenmux","modelId":"claude-sonnet-4-6","timestamp":"2024-01-15T09:30:01Z"}
{"type":"message","id":"e5f6g7h8","parentId":"a1b2c3d4","message":{"role":"user","content":"Explain the .jsonl format"}}
{"type":"message","id":"i9j0k1l2","parentId":"e5f6g7h8","message":{"role":"assistant","content":[{"type":"text","text":"The .jsonl format..."}]}}
{"type":"message","id":"m3n4o5p6","parentId":"i9j0k1l2","message":{"role":"toolResult","toolCallId":"call_abc","content":[{"type":"text","text":"Search results..."}]}}

The session header's parentSession field enables forking—creating new sessions that branch from any point in an existing conversation.

---

Entry Normalization: From Storage Types to UI Types

Pi Web maintains two parallel arrays that connect raw storage to rendered interface:

• messages[] — UI-ready message objects
• entryIds[] — original entry IDs from the .jsonl file

This parallelism in buildSessionContext (from lib/session-reader.ts#L29) enables precise fork and navigation operations that reference underlying entry IDs.

The Normalization Pipeline

Step 1: Read raw entries

In lib/session-reader.ts#L24-L66, Pi Web calls the Pi SDK's SessionManager.open(...).getEntries() to retrieve all entries from disk.

Step 2: Build entry lookup map

A byId Map provides O(1) access to any entry by its ID.

Step 3: Compute SDK session context

piBuildSessionContext (Pi SDK) determines the logical message tree, current thinking level, and active model.

Step 4: Transform to UI messages

entryToUiMessage (from lib/session-reader.ts#L30-L44) converts each entry, applying tool-call normalization and filtering (deferring thinking blocks, omitting large base-64 images).

Step 5: Return SessionContext

The final structure contains messages[], entryIds[], thinkingLevel, and model for UI consumption.

---

Tool-Call Field Normalization in normalize.ts

The most complex normalization handles tool-call format variations between the Pi SDK and Pi Web's UI requirements. The SDK may emit either modern or legacy field names—normalize.ts unifies them.

Field Mapping Logic

Source Field (SDK)	Target Field (UI)	Resolution Strategy
toolCallId or id	toolCallId	Prefer toolCallId, fall back to id, default to ""
toolName or name	toolName	Prefer toolName, fall back to name, default to ""
input or arguments	input (object)	Select first existing object field, default to {}

This logic appears in lib/normalize.ts#L31-L35 and is exposed through normalizeToolCalls (lib/normalize.ts#L57-L59).

Normalization Usage

During session loading:

// lib/session-reader.ts
import { normalizeToolCalls } from "./normalize";

function entryToUiMessage(entry: SessionEntry): UIMessage {
  const normalized = normalizeToolCalls(entry.message);
  // ... additional processing
  return { ...normalized, uiSpecificFields };
}

During streaming:

normalizeStreamingToolCalls (lib/normalize.ts#L61-L63) handles live tool-call updates, merging partial streaming input into normalized structures.

Complete Normalization Example

import { normalizeToolCalls } from "./normalize";

const sdkMessage = {
  role: "assistant",
  content: [{
    type: "toolCall",
    id: "call_abc123",           // legacy field name
    name: "read_file",           // legacy field name
    arguments: { path: "/etc/config" }  // legacy field name
  }]
};

const uiMessage = normalizeToolCalls(sdkMessage);
// Result: content[0] becomes { toolCallId: "call_abc123", toolName: "read_file", input: { path: "/etc/config" } }

---

Working with entryIds[] for Session Operations

The parallel entryIds[] array enables precise session manipulation:

// Fork at a specific message in the UI
function handleForkAtMessage(messageIndex: number) {
  const entryId = sessionContext.entryIds[messageIndex];
  
  await fetch(`/api/agent/${sessionId}/fork`, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ entryId })  // References exact .jsonl line
  });
}

This mechanism appears in the fork button logic within components/ChatWindow.tsx and throughout navigation features.

---

Loading and Building Session Context

Complete workflow for programmatic session access:

import { resolveSessionPath, buildSessionContext, getSessionEntries } from "./lib/session-reader";

async function loadSessionForDisplay(sessionId: string) {
  // Resolve file path from session ID
  const filePath = await resolveSessionPath(sessionId);
  if (!filePath) throw new Error(`Session not found: ${sessionId}`);

  // Read all raw entries from .jsonl
  const entries = getSessionEntries(filePath);

  // Build complete UI context
  const context = buildSessionContext(entries, /*leafId=*/ null);
  
  return {
    messages: context.messages,      // UI-ready chat messages
    entryIds: context.entryIds,      // Original entry references
    model: context.model,            // Active model configuration
    thinkingLevel: context.thinkingLevel  // Current thinking depth
  };
}

Source: lib/session-reader.ts#L38-L66

---

Key Implementation Files

File	Responsibility	Critical Exports
lib/session-reader.ts	.jsonl I/O, context building	buildSessionContext, entryToUiMessage, getSessionEntries
lib/normalize.ts	SDK-to-UI type conversion	normalizeToolCalls, normalizeStreamingToolCalls
lib/types.ts	TypeScript interfaces	SessionEntry, AgentMessage, SessionContext, ToolCallContent
hooks/useAgentSession.ts	React integration	Session state management for components

The AGENTS.md document in the repository root provides the canonical specification for the .jsonl format and serves as the contract between Pi SDK writers and Pi Web readers.

---

Summary

• .jsonl format: Line-delimited JSON with typed entries (session, model_change, message, compaction, session_info) enabling append-only session storage and fork support
• Entry normalization: Parallel entryIds[] and messages[] arrays maintain bidirectional links between storage and UI
• Tool-call normalization: lib/normalize.ts reconciles toolCallId/id, toolName/name, and input/arguments field variations
• Context building: buildSessionContext orchestrates reading, SDK context computation, and UI message generation

---

Frequently Asked Questions

What is the purpose of the parentSession field in the session header?

The parentSession field contains an absolute path to another .jsonl file when the current session is a fork of an existing conversation. This enables Pi Web to trace conversation lineage and support branching workflows where users can explore alternative paths from any point in chat history.

How does Pi Web handle different tool-call field names from the Pi SDK?

The normalizeToolCalls function in lib/normalize.ts applies a priority-based resolution: it prefers modern field names (toolCallId, toolName, input) but falls back to legacy alternatives (id, name, arguments) when needed. This ensures compatibility across SDK versions without breaking existing session files.

Why are entryIds[] and messages[] kept as separate parallel arrays rather than a single combined structure?

Separating these arrays preserves type safety and performance. The messages[] array contains UI-optimized objects with derived fields and filtered content, while entryIds[] provides direct references to raw .jsonl lines needed for fork and navigation operations. This design avoids duplicating heavy message content in ID lookups and keeps the boundary between storage identity and display representation explicit.

What happens when a session grows too large?

The compaction entry type ("type":"compaction") marks historical summarization events. When Pi Web or the Pi SDK compacts a session, it writes a compaction entry containing a text summary, the firstKeptEntryId marking the new history boundary, and tokensBefore indicating the token count reduction. This allows pruning of early messages while maintaining conversation continuity.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}