How MemoryReflector Condenses Agent memory.md Files Using Atomic-Swap

The MemoryReflector is a background janitor service in Munder Difflin that compresses oversized agent memory files using AI summarization and atomically swaps them to guarantee crash safety and data integrity.

The MemoryReflector is a critical component in the chaitanyagiri/munder-difflin repository that manages storage bounds for agent memory. Operating exclusively within the Electron main process to ensure filesystem permissions on macOS, it prevents data loss during condensation operations through a rigorous six-phase pipeline.

What Is the MemoryReflector?

The MemoryReflector implements the condense half of the janitor service. Its primary responsibility is maintaining bounded size constraints on agents/<id>/memory.md files while preserving essential information through intelligent summarization.

According to the source code header in src/main/reflect.ts (lines 1-22), this component requires main-process privileges to perform atomic filesystem operations safely across platforms.

The Six-Step Condense Pipeline

The reflector processes each agent's memory through a deterministic pipeline that prioritizes data safety over execution speed.

1. Trigger Detection with shouldCondense()

Every interval, the reflector scans the agents directory to identify condensation candidates. The shouldCondense() method (lines 77-84) returns true when a memory.md file exceeds a configurable byte-percentage of the global budget or contains too many ## sections.

2. Lossless Backup Creation

Before any modification, the system writes a lossless copy to hive/backups/<timestamp>/<id>/memory.md (lines 200-206). This guarantees that a failure at any subsequent step never results in data loss.

3. AI Summarization via runHiddenClaude()

The reflector invokes the headless Claude model (claude-haiku-4-5) using a deterministic system prompt (CONDENSE_SYSTEM). The prompt requests a JSON object containing a new condensed summary and any "hoisted" durable facts (lines 61-78). The actual execution occurs within runHiddenClaude() (lines 279-287).

4. Memory Reconstruction with rebuild()

The rebuild() method (lines 221-224) merges three distinct regions: the preserved "pinned" block, the newly generated condensed text, and the newest K verbatim recent sections. This creates a canonical three-region structure that maintains context continuity.

5. Verification Gate via verify()

Before committing changes, verify() (lines 271-299) enforces deterministic constraints:

• Correct three-region structure
• Non-empty condensed summary
• Size reduction exceeding 5%
• Preservation of every old pinned line
• Byte-exact integrity of retained recent sections

If any check fails, the file remains untouched and the system logs an abort event.

6. Atomic-Swap Write with atomicWrite()

Upon verification success, atomicWrite() (lines 328-336) implements POSIX atomic replacement:

function atomicWrite(path: string, text: string): void {
  const tmp = `${path}.tmp-${Math.random().toString(36).slice(2, 10)}`;
  writeFileSync(tmp, text, 'utf8');
  try {
    const fd = openSync(tmp, 'r+');
    try { fsyncSync(fd); } finally { closeSync(fd); }
  } catch { /* best-effort */ }
  renameSync(tmp, path);          // POSIX atomic replace
}

This pattern writes to a temporary sibling file, flushes the descriptor with fsyncSync, then renames the temporary file over the original path.

Why Atomic-Swap Guarantees Data Integrity

The atomic-swap pattern provides three critical safety properties:

• Crash safety: If the process crashes after temporary file creation but before rename, the original memory.md remains intact.
• Durability: The fsync operation guarantees the temporary file's contents are flushed to disk before the rename, satisfying POSIX atomic-replace semantics.
• Idempotence: The verifier ensures each new file is strictly smaller and structurally sound, preventing size regression across repeated runs.

Implementation Examples

Instantiating the MemoryReflector

import { MemoryReflector } from './reflect';

const reflector = new MemoryReflector(
  () => process.env.HIVE_HOME ?? null,          // getHome
  () => 'claude',                               // getCommand
  () => ({ MEMPALACE: '/path/to/palace' }),     // getMemoryEnv
  () => ({                                     // getSettings
    enabled: true,
    intervalMs: 300_000,
    byteTriggerPct: 90,
    sectionTrigger: 40,
    recentKeep: 5,
    minBytes: 16_384,
  }),
  (event) => console.log('log', event)          // appendLog
);
reflector.start();

Manual Condensation Trigger

Force immediate condensation for a specific agent bypassing the interval check:

// Condense only agent "alice" on demand
await reflector.reflectNow('alice');

Verification Logic Structure

The verify() function returns strict type discrimination for success or failure:

export function verify(args: {
  rebuilt: string; newBytes: number; oldBytes: number;
  oldPinnedLines: string[]; mergedPinned: string[];
  condensed: string; keep: Section[];
}): { ok: true } | { ok: false; reason: string } {
  // … (checks for structure, size, pinned preservation, recent integrity)
}

Summary

• The MemoryReflector in chaitanyagiri/munder-difflin implements crash-safe memory condensation for agent files using a six-phase pipeline.
• The process includes trigger detection, backup creation, AI summarization via claude-haiku-4-5, reconstruction, verification, and atomic-swap write.
• The atomicWrite() function in src/main/reflect.ts (lines 328-336) uses POSIX atomic rename to guarantee that the system never corrupts memory.md during power failures or crashes.
• The verify() function enforces strict structural and size constraints before permitting any file replacement, ensuring idempotent operation.

Frequently Asked Questions

What triggers the MemoryReflector to condense a memory file?

The shouldCondense() method evaluates two criteria: whether the file exceeds a configured byte-percentage of the global memory budget, or whether it contains more than the allowed number of ## sections. When either threshold is crossed, the condensation pipeline initiates immediately.

Why does MemoryReflector use atomic-swap instead of direct file overwrite?

Direct overwrites risk leaving a partially written file if the process crashes mid-operation. The atomicWrite() implementation writes to a temporary file, flushes to disk with fsyncSync, then performs a POSIX atomic renameSync. This ensures that readers always see either the complete old file or the complete new file, never a corrupted intermediate state.

What happens if verification fails during the condensation process?

If verify() detects structural errors, size increases, or missing pinned content, it returns { ok: false, reason: string } and the pipeline aborts. The original memory.md remains untouched, the temporary file is discarded, and the system logs the failure event without modifying the agent's memory.

Which AI model powers the MemoryReflector summarization?

The reflector calls the headless Claude CLI using the claude-haiku-4-5 model through runHiddenClaude() (lines 279-287). It uses a deterministic system prompt (CONDENSE_SYSTEM) to generate JSON containing condensed summaries and durable facts that survive the compression process.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}