# How Session Stores Persist and Retrieve Conversation History Efficiently in ClosedClaw

> Discover how ClosedClaw's session stores use atomic JSON5 files, in-memory caching, and file locking for crash-resilient persistence and fast retrieval of conversation history.

- Repository: [aSafeLobotomy/closedclaw](https://github.com/asafelobotomy/closedclaw)
- Tags: internals
- Published: 2026-02-25

---

**ClosedClaw implements session stores as atomic JSON5 files with in-memory TTL caching and file-based locking to ensure crash-resilient persistence and millisecond-scale retrieval of conversation metadata.**

Session stores form the backbone of conversation state management in ClosedClaw, mapping unique session keys to `SessionEntry` objects that track channel context, model preferences, and delivery metadata. Unlike monolithic databases, this lightweight file-based architecture prioritizes **fast reads** through aggressive caching while guaranteeing **safe concurrent writes** via atomic file operations and per-store lock files.

## Architecture Overview of ClosedClaw Session Stores

### File Format and Data Structure

Session data persists as JSON5 files (default: [`sessions.json`](https://github.com/asafelobotomy/closedclaw/blob/main/sessions.json)) located at configurable store paths. Each file contains a flat mapping of session keys to `SessionEntry` objects defined in [`src/config/sessions/types.ts`](https://github.com/asafelobotomy/closedclaw/blob/main/src/config/sessions/types.ts):

```typescript
// src/config/sessions/types.ts
export type SessionEntry = {
  sessionId: string;
  updatedAt: number;
  channel?: string;
  lastChannel?: string;
  deliveryContext?: unknown;
  // ... additional metadata fields
};

```

The JSON5 format allows human-readable storage with comments and relaxed syntax, while the flat structure enables O(1) key lookups after parsing.

### Separation of Metadata and Transcripts

The session store deliberately maintains **metadata only**—channel identifiers, timestamps, and routing context. The actual conversation transcript lives in separate per-session JSONL files (e.g., `session-<key>.jsonl`) managed by the gateway and memory modules. This separation prevents the session store from bloating as conversation history grows, keeping read/write operations constant-time regardless of message volume.

## Atomic Persistence Mechanisms

### Lock-Based Concurrency Control

ClosedClaw prevents write corruption through file-based locking implemented in [`src/config/sessions/store.ts`](https://github.com/asafelobotomy/closedclaw/blob/main/src/config/sessions/store.ts). Before any mutation, the system acquires a per-store lock file (`<storePath>.lock`) using `fs.promises.open` with the `wx` flag:

```typescript
// src/config/sessions/store.ts – lines 90-120
while (true) {
  try {
    // Attempt exclusive lock creation
    await fs.promises.open(lockPath, "wx");
    break;
  } catch (err) {
    // Handle ENOENT, EEXIST, or stale locks
    if (isStaleLock(lockPath)) {
      await fs.promises.unlink(lockPath);
    }
    await delay(pollInterval);
  }
}

```

Stale locks older than 30 seconds (configurable via `staleMs`) are automatically evicted to recover from crashed processes. The lock always releases in a `finally` block, ensuring cleanup even if the write operation throws.

### Crash-Resilient File Operations

All writes follow an atomic rename pattern to prevent partial file corruption. The `saveSessionStoreUnlocked` function writes to a temporary file first, then performs an atomic rename:

```typescript
// src/config/sessions/store.ts – lines 24-31
const tmp = `${storePath}.${process.pid}.${crypto.randomUUID()}.tmp`;
await fs.promises.writeFile(tmp, json, { 
  mode: 0o600, 
  encoding: "utf-8" 
});
await fs.promises.rename(tmp, storePath);

```

This guarantees that readers always see either the complete previous state or the complete new state, never a truncated intermediate. File permissions are set to `0o600` (owner read/write only) for security.

## High-Performance Retrieval with Caching

### TTL-Based Cache Invalidation

The retrieval path optimizes for repeated reads through an in-memory cache with configurable Time-To-Live (TTL). The `loadSessionStore` function in [`src/config/sessions/store.ts`](https://github.com/asafelobotomy/closedclaw/blob/main/src/config/sessions/store.ts) implements a two-tier check:

1. **Cache existence**: Verify the store path exists in `SESSION_STORE_CACHE`
2. **Freshness check**: Compare cached timestamp against `SESSION_STORE_TTL_MS` (default 45 seconds, configurable via `ClosedClaw_SESSION_CACHE_TTL_MS`)

```typescript
// src/config/sessions/store.ts – lines 13-30
export function loadSessionStore(storePath: string, opts = {}): Record<string, SessionEntry> {
  // 1️⃣ Check in-memory cache
  // 2️⃣ Verify file mtime matches cached version
  // 3️⃣ If cache miss or stale, read file, parse JSON5, normalize
}

```

Cache invalidation occurs proactively: `invalidateSessionStoreCache(storePath)` clears the cache entry immediately before any write operation, ensuring subsequent reads load the latest disk state.

### Deep Cloning for Cache Safety

To prevent cache pollution, the system returns **deep-cloned copies** of cached objects using `structuredClone`. This ensures that consumers can mutate session entries (e.g., updating `totalTokens` or `channel`) without corrupting the cached reference:

```typescript
// Conceptual implementation detail from store.ts
return structuredClone(cachedEntry);

```

The combination of TTL caching and defensive copying yields sub-millisecond retrieval times for hot sessions while maintaining data integrity across concurrent operations.

## Version-Tolerant Data Migrations

ClosedClaw session stores implement forward-compatible migrations that run automatically on every load. These idempotent transformations handle schema evolution without breaking existing deployments:

| Migration | Transformation | Code Location |
|-----------|---------------|---------------|
| Provider rename | `provider` → `channel` | [`src/config/sessions/store.ts`](https://github.com/asafelobotomy/closedclaw/blob/main/src/config/sessions/store.ts) lines 46-50 |
| Last provider rename | `lastProvider` → `lastChannel` | lines 51-55 |
| Room normalization | `room` → `groupChannel` (or deletion) | lines 55-61 |
| Channel fallback | Unknown channel → `gtk-gui` | lines 66-81 |

Because migrations run during the `loadSessionStore` call, the in-memory cache always contains normalized, up-to-date structures regardless of the file's age.

## Practical Implementation Examples

The following example demonstrates loading sessions and performing atomic updates using the public API:

```typescript
import { loadSessionStore, updateSessionStoreEntry } from "./src/config/sessions/store.js";

const storePath = "/home/user/.closedclaw/sessions.json";

// Load all sessions (cached if recent)
const sessions = loadSessionStore(storePath);
console.log("Active sessions:", Object.keys(sessions));

// Update a single session atomically
await updateSessionStoreEntry({
  storePath,
  sessionKey: "main",
  update: async (entry) => ({
    // Change the active channel
    channel: "webchat",
    // Increment a custom counter
    totalTokens: (entry.totalTokens ?? 0) + 100,
  }),
});

```

The `updateSessionStoreEntry` helper automatically handles locking, cache invalidation, and atomic file writes, providing a race-condition-free interface for session mutations.

## Summary

- **ClosedClaw session stores** use JSON5 files with atomic rename operations to guarantee crash-resilient persistence of conversation metadata.
- **Concurrent safety** is enforced through per-store lock files with automatic stale-lock eviction, preventing corruption from simultaneous writes.
- **High-speed retrieval** relies on an in-memory TTL cache with `structuredClone` isolation, enabling sub-millisecond access to hot sessions.
- **Schema evolution** is handled through automatic, idempotent migrations that normalize legacy field names on every load.
- **Separation of concerns** keeps lightweight metadata in the session store while conversation transcripts reside in separate JSONL files, ensuring constant-time operations regardless of message history length.

## Frequently Asked Questions

### What file format does ClosedClaw use for session stores?

ClosedClaw persists session data as **JSON5 files** (default: [`sessions.json`](https://github.com/asafelobotomy/closedclaw/blob/main/sessions.json)). JSON5 was chosen over standard JSON to support human-readable configuration with comments and relaxed syntax, while maintaining machine-parseable structure. The files store a flat mapping of session keys to `SessionEntry` objects containing metadata like `sessionId`, `updatedAt`, and `channel`.

### How does ClosedClaw handle concurrent writes to session stores?

The system implements **file-based locking** using per-store lock files (`<storePath>.lock`). Before writing, ClosedClaw attempts to create the lock exclusively using `fs.promises.open` with the `wx` flag. If the lock exists, the process polls until the lock releases or becomes stale (older than 30 seconds), at which point it forcibly removes the orphaned lock. All writes occur within a `finally` block that guarantees lock cleanup, preventing deadlocks even if the process crashes mid-write.

### What is the difference between session metadata and conversation transcripts?

**Session metadata** includes routing information, channel identifiers, model preferences, and timestamps—lightweight data stored in the JSON5 session store. **Conversation transcripts** contain the actual message history (user inputs, assistant responses, token counts) and persist in separate per-session JSONL files (e.g., `session-<key>.jsonl`). This architectural separation ensures that session store operations remain constant-time regardless of conversation length, while transcripts can grow indefinitely without impacting session retrieval performance.

### How does the session store cache prevent data corruption?

The cache employs **defensive copying** via `structuredClone` to isolate cached objects from consumer mutations. When `loadSessionStore` returns a cache hit, it returns a deep clone rather than the original reference. This prevents accidental modifications to the cached entry from corrupting the store for subsequent reads. Additionally, the cache implements **proactive invalidation**: every write operation calls `invalidateSessionStoreCache` immediately before modifying the file, ensuring that subsequent reads always load the latest disk state rather than stale cached data.