How Pi Web Reads and Parses Session Files From .jsonl Format

Pi Web reads .jsonl session files by streaming the header line, loading entries through the SessionManager SDK, and converting raw SDK entries into UI-ready messages using type-specific transforms in lib/session-reader.ts.

The Pi Web client transforms line‑delimited JSON session files into rich, typed objects for the chat interface. Each session is stored as a .jsonl file where the first line contains metadata and subsequent lines hold messages, tool calls, compactions, and branch summaries. This article explains the complete parsing pipeline implemented in agegr/pi-web.

---

Reading the Session Header

The header contains session metadata: id, cwd, timestamps, and configuration. Pi Web extracts it without loading the entire file.

readSessionHeader(filePath) in lib/session-reader.ts performs this in four steps:

1. Opens the file and reads up to the first newline (64 KB max buffer)
2. Parses the line as JSON
3. Validates that type === "session"
4. Returns the header object or null on failure

import { readSessionHeader } from '@/lib/session-reader';

const header = readSessionHeader('/path/to/session.jsonl');
// { id: 'sess_abc123', type: 'session', cwd: '/project', created: 1715000000, ... }

Source: lib/session-reader.ts, lines 88–116.

---

Loading Session Entries

After the header, Pi Web fetches all subsequent line objects as entries. The getSessionEntries(filePath) function delegates to the SDK's SessionManager:

import { getSessionEntries } from '@/lib/session-reader';

const entries = getSessionEntries('/path/to/session.jsonl');
// Array of SDK entry objects, cast to SessionEntry type

The implementation (lines 21–24) calls SessionManager.open(filePath) to obtain a Session instance, then invokes its getEntries() method. Raw SDK entries are cast to the local SessionEntry type for type safety downstream.

---

Building the UI Context

Raw entries alone don't determine which messages belong to the active conversation branch. buildSessionContext(entries, leafId, options) solves this by:

1. Creating an entry ID map from the entries array
2. Calling piBuildSessionContext and piBuildContextEntries from @earendil-works/pi-coding-agent to resolve the active branch
3. Attaching thinkingLevel and model from the SDK context

import { buildSessionContext } from '@/lib/session-reader';

const context = buildSessionContext(entries, null, { deferThinking: true });
// {
//   messages: AgentMessage[],
//   entryIds: string[],
//   thinkingLevel: number,
//   model: string
// }

Source: lib/session-reader.ts, lines 26–34.

---

Converting Entries to UI Messages

The core transformation happens in entryToUiMessage(entry, options), which switches on entry.type to produce AgentMessage objects:

Entry Type	UI Output	Behavior
"message"	Standard chat message	Normalizes tool calls, strips base64 images, defers "thinking" blocks when options.deferThinking is true
"compaction"	Custom compaction message	Represents a checkpoint where older messages were summarized
"branch_summary"	Summary message	Displays a user-visible description of a side branch
"custom_message"	Passthrough payload	Forwards arbitrary UI extensions unchanged
(other)	null	Metadata entries filtered from display

Tool call normalization runs through normalizeToolCalls (defined in lib/normalize.ts) to ensure payload structure matches the UI schema. Image stripping prevents large base64 strings from bloating the initial render.

Source: lib/session-reader.ts, lines 15–72.

---

Complete Usage Example

Here's the full pipeline from session ID to rendered messages:

import {
  resolveSessionPath,
  readSessionHeader,
  getSessionEntries,
  buildSessionContext
} from '@/lib/session-reader';

// 1. Resolve absolute path (cached via RPC manager)
const path = await resolveSessionPath('sess_abc123');
if (!path) throw new Error('Session not found');

// 2. Read metadata header
const header = readSessionHeader(path);

// 3. Load all entries
const entries = getSessionEntries(path);

// 4. Build context for current branch
const context = buildSessionContext(entries, /* leafId */ null, {
  deferThinking: true  // Hide "thinking" blocks until expanded
});

// context.messages now ready for React components

---

Key Files in the Pipeline

File	Purpose
lib/session-reader.ts	Header parsing, entry extraction, entryToUiMessage transforms
lib/normalize.ts	Tool call payload normalization
lib/session-file-references.ts	High-level helpers: getSessionEntries, resolveSessionPath
lib/rpc-manager.ts	Path caching and cache invalidation
app/api/sessions/[id]/context/route.ts	API endpoint exposing buildSessionContext to the frontend

---

Summary

• Header extraction: readSessionHeader streams the first line (max 64 KB) as JSON
• Entry loading: getSessionEntries uses SessionManager.open → getEntries() from the SDK
• Branch resolution: buildSessionContext delegates to piBuildSessionContext SDK helpers
• Type conversion: entryToUiMessage maps 4 entry types to UI messages, filters metadata
• On-disk format unchanged: All transformations happen in-memory for the frontend

---

Frequently Asked Questions

How does Pi Web handle large session files without loading everything into memory?

The header reader limits its buffer to 64 KB, and entry loading streams through the SDK's SessionManager. The UI context builder only processes entries reachable from the active leaf, skipping unreachable branches unless explicitly requested.

What happens to unknown entry types in the .jsonl file?

entryToUiMessage returns null for any type not explicitly handled (message, compaction, branch_summary, custom_message). These entries are filtered from the messages array but remain in entryIds for reference consistency.

Where does the SDK's SessionManager come from?

Pi Web imports piBuildSessionContext, piBuildContextEntries, and SessionManager from @earendil-works/pi-coding-agent, the underlying SDK that manages the .jsonl persistence format.

Can I defer thinking blocks without rebuilding the context?

The deferThinking option only affects the initial transform in entryToUiMessage. To toggle visibility later, modify the thinkingLevel property on existing messages client-side; the context does not need rebuilding.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}