How Agent Sessions Are Managed and Persisted to Disk in Craft Agents

Craft Agents stores every agent session on the local filesystem under the workspace's sessions/ folder, using a JSON-Lines file for the session header and messages alongside subdirectories for attachments and plans, with all writes handled through an atomic persistence queue to ensure data integrity.

Craft Agents is an open-source AI agent framework that maintains stateful conversations through a robust filesystem-backed session model. Understanding how agent sessions are managed and persisted to disk in Craft Agents is essential for building reliable applications that survive process restarts and support concurrent access. The implementation leverages atomic file operations, path sanitization, and a debounced write queue to balance durability with performance.

Directory Layout and Session Structure

Each session is represented as a directory inside {workspaceRootPath}/sessions/. The core path utilities reside in packages/shared/src/sessions/storage.ts.

The directory structure follows this schema:

{workspaceRootPath}/sessions/
    └─ {sessionId}/
        ├─ session.jsonl      ← header + messages (JSONL)
        ├─ attachments/       ← file attachments from tools
        ├─ plans/            ← markdown plan files (Safe Mode)
        ├─ data/             ← transform_data tool output (JSON)
        ├─ long_responses/   ← full tool results that were trimmed
        └─ downloads/        ← binary files from API sources (PDFs, images, …)

The function getSessionPath() constructs these paths after sanitizing the session ID via sanitizeSessionId() to prevent path-traversal attacks. This defense-in-depth measure ensures that malicious session IDs cannot escape the designated workspace directory. The JSON-Lines file itself is accessed through getSessionFilePath().

Creating New Agent Sessions

Session creation is handled by createSession() in packages/shared/src/sessions/storage.ts. This function orchestrates the initialization of all on-disk resources through the following steps:

1. Ensure base directory: Calls ensureSessionsDir() to verify the top-level sessions folder exists.
2. Generate unique ID: Uses generateSessionId() (delegating to generateUniqueSessionId()) to create a human-readable, filesystem-safe identifier.
3. Create subdirectories: Invokes ensureSessionDir() to atomically create the session folder and all tool subdirectories (attachments, plans, data, etc.).
4. Set immutable working directory: Records the sdkCwd (SDK working directory) in the session header, guaranteeing that transcript files remain locatable even if the process changes working directories later.
5. Persist initial state: Calls saveSession() to write an empty session JSON-Lines file, ensuring the session is immediately discoverable by listSessions().

import { createSession } from '@craft-agent/shared/sessions';

const workspaceRoot = '/Users/alice/.craft-agent';
const session = await createSession(workspaceRoot, {
  name: 'Research on AI safety',
  model: 'claude-2',
});
// Session folder: /Users/alice/.craft-agent/sessions/260111-swift-river/

Atomic Persistence and the Write Queue

All disk writes are funneled through a persistence queue (sessionPersistenceQueue) defined in packages/shared/src/sessions/persistence-queue.ts and exported via storage.ts.

When saveSession() is invoked, it enqueues the session state and immediately flushes the queue. This design debounces rapid successive writes—such as multiple messages arriving in quick succession—by batching them into a single background thread operation. The queue performs atomic file writes by:

• Writing to a temporary .tmp file first
• Renaming the temporary file to session.jsonl to replace the old version atomically
• Cleaning up orphaned .tmp files during listing operations

The JSON-Lines format stores the session header on line 1, with each subsequent line containing a single message object. This structure enables listSessions() to read only the first line (via readSessionHeader()) when building session metadata lists, significantly improving performance for workspaces with large conversation histories.

Loading Sessions and Listing Metadata

Retrieving session data is handled by distinct functions optimized for different access patterns:

• loadSession(): Reads the complete JSON-Lines file using readSessionJsonl() and returns a fully hydrated StoredSession object including all messages.
• listSessions(): Iterates over subdirectories in the sessions/ folder, reads only the header from each session.jsonl, and constructs SessionMetadata objects sorted by lastUsedAt.

Specialized helper functions provide filtered views without loading full session data:

• listActiveSessions()
• listInboxSessions()
• listCompletedSessions()

These utilities examine the header fields (status, archive flags) to return relevant subsets, making UI navigation responsive even with thousands of historical sessions.

Updating Session Metadata and Lifecycle State

Craft Agents provides granular functions for modifying session metadata, all of which internally call saveSession() to ensure disk synchronization:

• updateSessionSdkId(): Associates the session with an external SDK identifier.
• updateSessionMetadata(): Modifies mutable fields including name, status, labels, and workingDirectory.
• flagSession() / unflagSession(): Toggles the isFlagged boolean for UI bookmarking.
• setSessionStatus(): Transitions the session between states (active, completed, etc.).
• archiveSession() / unarchiveSession(): Manages archival status and timestamps (archivedAt).
• clearSessionMessages(): Removes all message history and resets token usage counters, supporting the /clear command functionality.

Each function performs an atomic read-modify-write cycle through the persistence queue, preventing race conditions during concurrent updates.

Handling Plans and File Attachments

Tools that generate files store them in dedicated subdirectories created during session initialization:

• Plans: Stored as Markdown files in plans/. The utility savePlanToFile() serializes plan objects using formatPlanAsMarkdown(), while listPlanFiles() retrieves available plans with modification timestamps.
• Attachments: Binary files from tools (screenshots, documents) reside in attachments/. Before writing, tools call ensureAttachmentsDir() to verify the directory exists, though this is typically redundant since ensureSessionDir() creates all subdirectories upfront.

These directories are created with the session folder and persist for the lifetime of the session, ensuring that long-running agents retain access to generated artifacts across process restarts.

Session Cleanup and Retention Policies

Craft Agents provides explicit functions for lifecycle management:

• deleteSession(): Recursively removes the entire session directory tree, including all attachments and plan files.
• deleteOldArchivedSessions(): Implements retention policies by deleting archived sessions older than a configurable threshold. This function checks the archivedAt timestamp when available, falling back to lastUsedAt for legacy sessions.

These operations bypass the persistence queue since they operate on the filesystem directly rather than modifying JSON-Lines content.

Security and Concurrency Safeguards

The session storage layer implements multiple safety mechanisms:

Path Sanitization: Every public API function that accepts a session ID invokes sanitizeSessionId() to strip or escape characters that could enable directory traversal, providing defense-in-depth even if upstream validation fails.

Atomic Writes: The persistence queue's use of temporary files and atomic renames ensures that session.jsonl is never in a partially-written state visible to readers. If a crash occurs during write, the existing file remains intact and orphaned .tmp files are pruned during the next listing operation.

Permission Modes: Sessions can be initialized with a permissionMode stored in the header that restricts which files tools may read or write. The tool sandbox respects these constraints, preventing agents from accessing sensitive files outside their designated workspace.

Summary

• Craft Agents persists agent sessions to the local filesystem under {workspace}/sessions/{sessionId}/ using a JSON-Lines format for message history.
• The createSession() function in packages/shared/src/sessions/storage.ts generates sanitized IDs, creates subdirectories for attachments/plans, and sets an immutable sdkCwd.
• All writes pass through sessionPersistenceQueue in persistence-queue.ts, which provides debounced, atomic file operations using temporary files and renames.
• listSessions() optimizes performance by reading only JSON-Lines headers, while loadSession() retrieves complete conversation histories.
• Metadata updates (updateSessionMetadata, flagSession, archiveSession, etc.) are atomic and synchronized to disk immediately.
• Safety measures include path sanitization (sanitizeSessionId), atomic file replacement, and configurable permission modes for tool sandboxing.

Frequently Asked Questions

Where are Craft Agents session files stored on disk?

Session files are stored in a sessions/ subdirectory within the configured workspace root. Each session receives its own folder named after the sanitized session ID, containing a session.jsonl file and subdirectories for attachments, plans, and downloads. The path construction logic resides in packages/shared/src/sessions/storage.ts.

How does Craft Agents prevent data corruption during concurrent writes?

Craft Agents uses an atomic persistence queue (sessionPersistenceQueue) that serializes all write operations. When saveSession() is called, the data is written to a temporary .tmp file and then renamed to session.jsonl. This ensures that readers always see a complete, valid file state, even if the process crashes during the write operation.

What is the difference between session.jsonl and the subdirectories like attachments/ and plans/?

The session.jsonl file contains structured data: the session header (metadata, configuration) on line 1 and each subsequent line represents a message in the conversation. The subdirectories (attachments/, plans/, data/) store binary or large text files generated by tools—such as PDFs, images, or Markdown plan documents—that would be inefficient to embed in the JSON-Lines format. These files are referenced by path from the messages.

How can I programmatically delete old archived sessions?

Use the deleteOldArchivedSessions() function exported from packages/shared/src/sessions/storage.ts. This function accepts a workspace path and a retention threshold in days, then removes session directories where archivedAt (or lastUsedAt as a fallback) exceeds the specified age. This is typically invoked by a maintenance job or cleanup cron task.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}