# How Craft-Agents Persists Sessions and Stores Message History: A Technical Deep Dive

> Discover how Craft-Agents persists sessions and stores message history using JSON Lines and atomic writes. Ensure data integrity with portable paths for seamless migration.

- Repository: [Craft Docs/craft-agents-oss](https://github.com/lukilabs/craft-agents-oss)
- Tags: deep-dive
- Published: 2026-04-18

---

**Craft-Agents persists session data and message history to the local file system using a JSON Lines format within workspace-specific directories, implementing atomic writes and portable paths to ensure data integrity across machine migrations.**

Craft-Agents implements a robust, file-based persistence layer that stores every user session directly within the workspace directory. The system uses a specialized JSON Lines format for message history and implements atomic write operations to prevent data corruption during crashes.

## Directory Layout and Session Storage Structure

### The Sessions Folder Hierarchy

When a workspace initializes, Craft-Agents creates a `sessions/` folder at the workspace root. The system generates a unique session identifier for each conversation and creates a dedicated subdirectory structure:

```

workspace/
└── sessions/
    └── <session-id>/
        ├── session.jsonl
        ├── attachments/
        ├── plans/
        ├── data/
        ├── long_responses/
        └── downloads/

```

Each subfolder serves a specific purpose. The `session.jsonl` file contains the session header and complete message history. The `attachments/` directory stores binary files uploaded by users or generated by tools. Agent plans reside in `plans/`, while the `data/` folder holds JSON outputs from the `transform_data` tool. Long responses that were truncated for UI display are preserved in `long_responses/`, and external downloads (PDFs, images, archives) go into `downloads/`.

### Path Resolution Utilities

The helper functions in [`packages/shared/src/sessions/storage.ts`](https://github.com/lukilabs/craft-agents-oss/blob/main/packages/shared/src/sessions/storage.ts) manage these paths. The `ensureSessionDir` function creates the directory structure, while `getSessionPath` and `getSessionFilePath` generate absolute paths for session files. These utilities ensure consistent path resolution across different operating systems and workspace locations.

## Message History Format and JSON Lines Implementation

### The session.jsonl File Structure

Craft-Agents stores message history in a JSON Lines format within `session.jsonl`. This format provides several advantages: append-only writes, easy parsing of partial files, and resilience against corruption.

The file structure follows a strict line-based protocol:

```

{"id":"260411-bright-river","workspacePath":"/Users/me/my-workspace",...}  // Line 1: SessionHeader
{"id":"msg-1","type":"user","timestamp":1234567890,"content":"Hello"}       // Line 2+: StoredMessage
{"id":"msg-2","type":"assistant","timestamp":1234567891,"content":"Hi there"}

```

The first line contains the `SessionHeader` with metadata including the session ID, workspace path, creation timestamps, current status, and token usage statistics. Subsequent lines contain `StoredMessage` objects with fields for message type, unique identifier, content, and timestamps.

### Reading and Writing JSON Lines

The [`packages/shared/src/sessions/jsonl.ts`](https://github.com/lukilabs/craft-agents-oss/blob/main/packages/shared/src/sessions/jsonl.ts) file implements the read/write logic. For performance optimization, the `readSessionHeader` function reads only the first 8KB of the file to extract metadata without parsing the entire message history. This enables fast session list rendering in the UI.

When loading complete sessions, `readSessionJsonl` expands portable path tokens, parses the header, and processes each message line using `parseMessagesResilient`. This resilient parsing tolerates partially written lines that might occur during system crashes, ensuring that valid messages are recovered even if the last write was interrupted.

## Atomic Persistence and Portable Paths

### The Atomic Write Pattern

Craft-Agents implements atomic file writes to prevent session corruption. The `writeSessionJsonl` function in [`packages/shared/src/sessions/jsonl.ts`](https://github.com/lukilabs/craft-agents-oss/blob/main/packages/shared/src/sessions/jsonl.ts) performs writes using a temporary file and rename operation:

1. The system writes the header and all messages to a temporary file with a `.tmp` extension
2. Once the write completes successfully, the function atomically renames the temporary file over the existing `session.jsonl`
3. This ensures that readers never see partially written data; they either see the complete old file or the complete new file

### Portable Path Tokens

To enable session portability across different machines, Craft-Agents converts absolute paths to portable tokens before writing. The `makeSessionPathPortable` function replaces the session's absolute directory path with the token `{{SESSION_PATH}}}`.

When loading sessions, `expandSessionPath` resolves these tokens back to absolute paths based on the current workspace location. This mechanism allows users to copy session directories between computers or share them via version control without path breakage.

### The Persistence Queue

The `SessionPersistenceQueue` singleton in [`packages/shared/src/sessions/persistence-queue.ts`](https://github.com/lukilabs/craft-agents-oss/blob/main/packages/shared/src/sessions/persistence-queue.ts) manages write operations to prevent race conditions and optimize disk I/O. The queue implements debouncing for rapid successive updates, coalescing multiple save requests into a single write operation.

The queue maintains a metadata signature to detect external modifications. If the `session.jsonl` file changes on disk between the queue's read and write operations (for example, through a configuration watcher), the system reconciles the changes to prevent data loss.

## Practical Code Examples

### Creating a New Session

```typescript
import { createSession } from '@/packages/shared/src/sessions/storage.ts';

const newSession = await createSession('/Users/me/my-workspace', {
  name: 'Research on AI',
  workingDirectory: '/Users/me/my-workspace',
});

console.log('Session ID:', newSession.id);

```

### Loading an Existing Session and Reading Message History

```typescript
import { loadSession } from '@/packages/shared/src/sessions/storage.ts';

const stored = loadSession('/Users/me/my-workspace', '260411-bright-river');

if (stored) {
  console.log('Header preview:', stored.preview);
  console.log('Messages count:', stored.messages.length);
}

```

### Appending a New Message and Persisting Changes

```typescript
import { loadSession, saveSession } from '@/packages/shared/src/sessions/storage.ts';
import type { StoredMessage } from '@/packages/shared/src/sessions/types.ts';

const sess = loadSession('/Users/me/my-workspace', '260411-bright-river');

if (sess) {
  const msg: StoredMessage = {
    id: crypto.randomUUID(),
    type: 'user',
    timestamp: Date.now(),
    content: 'Can you summarise the latest article?',
  };
  
  sess.messages.push(msg);
  await saveSession(sess);   // goes through the persistence queue
}

```

### Deleting a Session

```typescript
import { deleteSession } from '@/packages/shared/src/sessions/storage.ts';

const ok = deleteSession('/Users/me/my-workspace', '260411-bright-river');
console.log('Deleted?', ok);

```

## Summary

- **Craft-Agents** persists sessions to the local file system using a workspace-scoped `sessions/` directory that contains individual subfolders for each conversation.
- **Message history** is stored in `session.jsonl` using a JSON Lines format where line 1 contains the session header and subsequent lines contain individual messages.
- **Atomic writes** prevent corruption by writing to temporary files and performing atomic renames, ensuring readers never see partially written data.
- **Portable paths** using the `{{SESSION_PATH}}` token enable sessions to be moved between machines without breaking file references.
- **SessionPersistenceQueue** debounces rapid saves and tracks metadata signatures to prevent race conditions and external modification conflicts.

## Frequently Asked Questions

### How does Craft-Agents prevent data loss if the application crashes during a save?

The system implements atomic file writes in [`packages/shared/src/sessions/jsonl.ts`](https://github.com/lukilabs/craft-agents-oss/blob/main/packages/shared/src/sessions/jsonl.ts). When saving, it writes the entire session data to a temporary `.tmp` file first, then performs an atomic rename operation to replace the existing `session.jsonl`. This ensures that the file is never in a partially written state; readers either see the complete old version or the complete new version.

### Can I move a Craft-Agents session to another computer?

Yes. Craft-Agents uses portable path tokens to ensure session portability. Before writing to disk, absolute paths are converted to the `{{SESSION_PATH}}` token via `makeSessionPathPortable` in [`packages/shared/src/sessions/jsonl.ts`](https://github.com/lukilabs/craft-agents-oss/blob/main/packages/shared/src/sessions/jsonl.ts). When loading on a new machine, the system expands these tokens based on the current workspace location, allowing attachments and file references to resolve correctly regardless of the absolute path differences between machines.

### What is the SessionPersistenceQueue and why is it necessary?

The `SessionPersistenceQueue` in [`packages/shared/src/sessions/persistence-queue.ts`](https://github.com/lukilabs/craft-agents-oss/blob/main/packages/shared/src/sessions/persistence-queue.ts) is a singleton that manages all disk write operations for sessions. It prevents race conditions by serializing write requests and optimizes performance by debouncing rapid successive updates into single atomic writes. The queue also tracks metadata signatures to detect when external processes (like file watchers) modify the session file, preventing accidental overwrites of external changes.

### How does Craft-Agents handle large message histories efficiently?

For operations that only need session metadata (such as rendering a session list), Craft-Agents uses the `readSessionHeader` function in [`packages/shared/src/sessions/jsonl.ts`](https://github.com/lukilabs/craft-agents-oss/blob/main/packages/shared/src/sessions/jsonl.ts) to read only the first 8KB of the file, parsing just the header line without loading the entire message history. For full session loads, the `parseMessagesResilient` function tolerates partially written lines, ensuring that valid messages are recovered even if the last write operation was interrupted.