How Session Stores Persist and Retrieve Conversation History Efficiently in ClosedClaw

ClosedClaw implements session stores as atomic JSON5 files with in-memory TTL caching and file-based locking to ensure crash-resilient persistence and millisecond-scale retrieval of conversation metadata.

Session stores form the backbone of conversation state management in ClosedClaw, mapping unique session keys to SessionEntry objects that track channel context, model preferences, and delivery metadata. Unlike monolithic databases, this lightweight file-based architecture prioritizes fast reads through aggressive caching while guaranteeing safe concurrent writes via atomic file operations and per-store lock files.

Architecture Overview of ClosedClaw Session Stores

File Format and Data Structure

Session data persists as JSON5 files (default: sessions.json) located at configurable store paths. Each file contains a flat mapping of session keys to SessionEntry objects defined in src/config/sessions/types.ts:

// src/config/sessions/types.ts
export type SessionEntry = {
  sessionId: string;
  updatedAt: number;
  channel?: string;
  lastChannel?: string;
  deliveryContext?: unknown;
  // ... additional metadata fields
};

The JSON5 format allows human-readable storage with comments and relaxed syntax, while the flat structure enables O(1) key lookups after parsing.

Separation of Metadata and Transcripts

The session store deliberately maintains metadata only—channel identifiers, timestamps, and routing context. The actual conversation transcript lives in separate per-session JSONL files (e.g., session-<key>.jsonl) managed by the gateway and memory modules. This separation prevents the session store from bloating as conversation history grows, keeping read/write operations constant-time regardless of message volume.

Atomic Persistence Mechanisms

Lock-Based Concurrency Control

ClosedClaw prevents write corruption through file-based locking implemented in src/config/sessions/store.ts. Before any mutation, the system acquires a per-store lock file (<storePath>.lock) using fs.promises.open with the wx flag:

// src/config/sessions/store.ts – lines 90-120
while (true) {
  try {
    // Attempt exclusive lock creation
    await fs.promises.open(lockPath, "wx");
    break;
  } catch (err) {
    // Handle ENOENT, EEXIST, or stale locks
    if (isStaleLock(lockPath)) {
      await fs.promises.unlink(lockPath);
    }
    await delay(pollInterval);
  }
}

Stale locks older than 30 seconds (configurable via staleMs) are automatically evicted to recover from crashed processes. The lock always releases in a finally block, ensuring cleanup even if the write operation throws.

Crash-Resilient File Operations

All writes follow an atomic rename pattern to prevent partial file corruption. The saveSessionStoreUnlocked function writes to a temporary file first, then performs an atomic rename:

// src/config/sessions/store.ts – lines 24-31
const tmp = `${storePath}.${process.pid}.${crypto.randomUUID()}.tmp`;
await fs.promises.writeFile(tmp, json, { 
  mode: 0o600, 
  encoding: "utf-8" 
});
await fs.promises.rename(tmp, storePath);

This guarantees that readers always see either the complete previous state or the complete new state, never a truncated intermediate. File permissions are set to 0o600 (owner read/write only) for security.

High-Performance Retrieval with Caching

TTL-Based Cache Invalidation

The retrieval path optimizes for repeated reads through an in-memory cache with configurable Time-To-Live (TTL). The loadSessionStore function in src/config/sessions/store.ts implements a two-tier check:

1. Cache existence: Verify the store path exists in SESSION_STORE_CACHE
2. Freshness check: Compare cached timestamp against SESSION_STORE_TTL_MS (default 45 seconds, configurable via ClosedClaw_SESSION_CACHE_TTL_MS)

// src/config/sessions/store.ts – lines 13-30
export function loadSessionStore(storePath: string, opts = {}): Record<string, SessionEntry> {
  // 1️⃣ Check in-memory cache
  // 2️⃣ Verify file mtime matches cached version
  // 3️⃣ If cache miss or stale, read file, parse JSON5, normalize
}

Cache invalidation occurs proactively: invalidateSessionStoreCache(storePath) clears the cache entry immediately before any write operation, ensuring subsequent reads load the latest disk state.

Deep Cloning for Cache Safety

To prevent cache pollution, the system returns deep-cloned copies of cached objects using structuredClone. This ensures that consumers can mutate session entries (e.g., updating totalTokens or channel) without corrupting the cached reference:

// Conceptual implementation detail from store.ts
return structuredClone(cachedEntry);

The combination of TTL caching and defensive copying yields sub-millisecond retrieval times for hot sessions while maintaining data integrity across concurrent operations.

Version-Tolerant Data Migrations

ClosedClaw session stores implement forward-compatible migrations that run automatically on every load. These idempotent transformations handle schema evolution without breaking existing deployments:

Migration	Transformation	Code Location
Provider rename	provider → channel	src/config/sessions/store.ts lines 46-50
Last provider rename	lastProvider → lastChannel	lines 51-55
Room normalization	room → groupChannel (or deletion)	lines 55-61
Channel fallback	Unknown channel → gtk-gui	lines 66-81

Because migrations run during the loadSessionStore call, the in-memory cache always contains normalized, up-to-date structures regardless of the file's age.

Practical Implementation Examples

The following example demonstrates loading sessions and performing atomic updates using the public API:

import { loadSessionStore, updateSessionStoreEntry } from "./src/config/sessions/store.js";

const storePath = "/home/user/.closedclaw/sessions.json";

// Load all sessions (cached if recent)
const sessions = loadSessionStore(storePath);
console.log("Active sessions:", Object.keys(sessions));

// Update a single session atomically
await updateSessionStoreEntry({
  storePath,
  sessionKey: "main",
  update: async (entry) => ({
    // Change the active channel
    channel: "webchat",
    // Increment a custom counter
    totalTokens: (entry.totalTokens ?? 0) + 100,
  }),
});

The updateSessionStoreEntry helper automatically handles locking, cache invalidation, and atomic file writes, providing a race-condition-free interface for session mutations.

Summary

• ClosedClaw session stores use JSON5 files with atomic rename operations to guarantee crash-resilient persistence of conversation metadata.
• Concurrent safety is enforced through per-store lock files with automatic stale-lock eviction, preventing corruption from simultaneous writes.
• High-speed retrieval relies on an in-memory TTL cache with structuredClone isolation, enabling sub-millisecond access to hot sessions.
• Schema evolution is handled through automatic, idempotent migrations that normalize legacy field names on every load.
• Separation of concerns keeps lightweight metadata in the session store while conversation transcripts reside in separate JSONL files, ensuring constant-time operations regardless of message history length.

Frequently Asked Questions

What file format does ClosedClaw use for session stores?

ClosedClaw persists session data as JSON5 files (default: sessions.json). JSON5 was chosen over standard JSON to support human-readable configuration with comments and relaxed syntax, while maintaining machine-parseable structure. The files store a flat mapping of session keys to SessionEntry objects containing metadata like sessionId, updatedAt, and channel.

How does ClosedClaw handle concurrent writes to session stores?

The system implements file-based locking using per-store lock files (<storePath>.lock). Before writing, ClosedClaw attempts to create the lock exclusively using fs.promises.open with the wx flag. If the lock exists, the process polls until the lock releases or becomes stale (older than 30 seconds), at which point it forcibly removes the orphaned lock. All writes occur within a finally block that guarantees lock cleanup, preventing deadlocks even if the process crashes mid-write.

What is the difference between session metadata and conversation transcripts?

Session metadata includes routing information, channel identifiers, model preferences, and timestamps—lightweight data stored in the JSON5 session store. Conversation transcripts contain the actual message history (user inputs, assistant responses, token counts) and persist in separate per-session JSONL files (e.g., session-<key>.jsonl). This architectural separation ensures that session store operations remain constant-time regardless of conversation length, while transcripts can grow indefinitely without impacting session retrieval performance.

How does the session store cache prevent data corruption?

The cache employs defensive copying via structuredClone to isolate cached objects from consumer mutations. When loadSessionStore returns a cache hit, it returns a deep clone rather than the original reference. This prevents accidental modifications to the cached entry from corrupting the store for subsequent reads. Additionally, the cache implements proactive invalidation: every write operation calls invalidateSessionStoreCache immediately before modifying the file, ensuring that subsequent reads always load the latest disk state rather than stale cached data.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}