# How Munder Difflin Implements Persistent Memory for Agents: Architecture Deep-Dive

> Discover how Munder Difflin implements persistent memory for agents using plain-text files and a background MemPalace indexing pipeline for efficient semantic search and size management.

- Repository: [Chaitanya Giri/munder-difflin](https://github.com/chaitanyagiri/munder-difflin)
- Tags: architecture
- Published: 2026-08-28

---

**Munder Difflin implements persistent memory for agents using plain-text [`memory.md`](https://github.com/chaitanyagiri/munder-difflin/blob/main/memory.md) files stored per-agent, combined with a background MemPalace indexing pipeline for semantic search and automatic condensing to manage file size.**

Each agent in the Munder Difflin hive maintains its own long-term knowledge base in a simple markdown file. This design prioritizes durability, inspectability, and portability while adding powerful semantic search capabilities through an optional external CLI tool. The architecture spans agent creation, background indexing, memory maintenance, and runtime query interfaces.

## The Core Storage Model: Per-Agent memory.md Files

At the heart of Munder Difflin's persistent memory system lies a single convention: **every agent owns a [`memory.md`](https://github.com/chaitanyagiri/munder-difflin/blob/main/memory.md) file** located at `HIVE_ROOT/agents/<agent-id>/memory.md`. This file serves as the source of truth for an agent's accumulated knowledge and survives process restarts, crashes, and shutdowns without requiring a database.

When a new agent is initialized, the hive automatically provisions this file. In [[`src/main/hive.ts`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/hive.ts)](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/hive.ts), the `ensureAgent()` method creates the agent directory structure and writes an initial header to [`memory.md`](https://github.com/chaitanyagiri/munder-difflin/blob/main/memory.md):

```typescript
// Simplified excerpt from hive.ts agent provisioning
const agentDir = join(this.root, 'agents', agentId);
await mkdir(agentDir, { recursive: true });

const memoryPath = join(agentDir, 'memory.md');
if (!existsSync(memoryPath)) {
  await writeFile(memoryPath, `# ${agentName}\n\n## Memories\n\n`);

}

```

The plain-text format ensures the memory remains human-readable, version-control friendly, and accessible to standard Unix tools.

## The MemPalace Pipeline: From Markdown to Semantic Search

Raw markdown files enable persistence but not efficient search. Munder Difflin solves this through **MemPalace**, an external CLI tool that builds a searchable embedding index from all agent memories.

### Mining: Converting memory.md to Searchable Wings

The [`MemoryManager`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/memory.ts) class orchestrates the mining pipeline. Located in [[`src/main/memory.ts`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/memory.ts)](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/memory.ts), it periodically invokes `mempalace mine` to ingest each agent's memory into a shared **palace** directory:

```typescript
// From src/main/memory.ts – the mining operation
async mineNow(): Promise<void> {
  for (const agent of this.hive.listAgents()) {
    const agentDir = this.hive.agentDir(agent.id);
    const wing = agent.id; // Each agent gets its own "wing" in the palace
    
    await execa('mempalace', [
      'mine', agentDir,
      '--wing', wing,
      '--agent', agent.id,
      '--output', this.palaceDir
    ]);
  }
}

```

The manager schedules automatic mining every 10 minutes via `setInterval`, with additional on-demand mining triggered after significant memory updates.

### Controlling the Index: The .gitignore Filter

Not all files in an agent directory belong in the semantic index. The [`ensureMineIgnore()`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/memory.ts#L35) function in [[`src/main/memory.ts`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/memory.ts)](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/memory.ts) generates a per-agent `.gitignore` that excludes:

- Configuration files ([`settings.json`](https://github.com/chaitanyagiri/munder-difflin/blob/main/settings.json), [`cursor.json`](https://github.com/chaitanyagiri/munder-difflin/blob/main/cursor.json))
- Message queues (`inbox/`, `outbox/` JSON files)
- Temporary artefacts

This ensures the MemPalace index contains only meaningful memory content.

### Memory Status Tracking

The [`MemoryStatus`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/memory.ts#L55) interface in [[`src/main/memory.ts`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/memory.ts)](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/memory.ts) exposes operational state:

```typescript
interface MemoryStatus {
  palaceReady: boolean;      // Is MemPalace CLI available?
  lastMineTime: Date | null; // When did last mining complete?
  indexedAgents: string[];   // Which agents are in the index?
  totalEmbeddings: number;   // Approximate index size
}

```

## Memory Maintenance: Automatic Condensing with Reflector

Unbounded growth would eventually make [`memory.md`](https://github.com/chaitanyagiri/munder-difflin/blob/main/memory.md) files unwieldy. Munder Difflin addresses this through **memory condensing** implemented in [[`src/main/reflect.ts`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/reflect.ts)](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/reflect.ts).

The **Reflector** periodically scans agent memories and restructures oversized files into three regions:

| Region | Purpose |
|--------|---------|
| **Pinned** | Critical facts the agent must retain (manually marked) |
| **Condensed** | LLM-summarized older content |
| **Recent** | Unprocessed entries from the current session |

The condensing process uses a headless LLM to summarize historical entries while preserving semantic meaning. After condensation, the next mining operation picks up the compacted version, keeping the MemPalace index efficient.

## Runtime Access: Querying Persistent Memory

Agents and UI components access memory through two primary interfaces.

### The get_memory Tool

The renderer exposes a **`get_memory`** tool defined in [[`src/renderer/src/realtime/tools.ts`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/renderer/src/realtime/tools.ts)](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/renderer/src/realtime/tools.ts). This tool implements a **dual-path search strategy**:

```typescript
// Simplified from tools.ts
async function get_memory({ query, agentId }: MemoryQuery): Promise<MemoryResult> {
  // Path 1: Semantic search via MemPalace CLI (preferred)
  if (memoryManager.status.palaceReady) {
    const result = await execa('mempalace', [
      'search', query,
      '--wing', agentId ?? '*',
      '--limit', '5'
    ]);
    return parseSemanticResults(result.stdout);
  }
  
  // Path 2: Fallback to direct text search across all memory.md files
  return grepAllMemories(query);
}

```

When the MemPalace CLI is available, queries execute as **semantic memory** searches using vector embeddings. Otherwise, the system falls back to **full-text search** across all [`memory.md`](https://github.com/chaitanyagiri/munder-difflin/blob/main/memory.md) files.

### Direct File Access via IPC

The preload bridge in [[`src/preload/index.ts`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/preload/index.ts)](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/preload/index.ts) exposes synchronous memory operations:

| IPC Channel | Purpose |
|-------------|---------|
| `memoryWakeUp` | Initialize memory subsystem on app launch |
| `memoryStatus` | Retrieve current [`MemoryStatus`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/memory.ts#L55) |
| `hiveMemory` | Read a specific agent's [`memory.md`](https://github.com/chaitanyagiri/munder-difflin/blob/main/memory.md) directly |

```typescript
// Reading Alice's memory from the renderer process
const aliceMemory: string = await window.cth.hiveMemory('alice');
// Returns raw contents of agents/alice/memory.md

```

## Complete Usage Examples

### Creating an Agent with Persistent Memory

```typescript
import { Hive } from './main/hive';

const hive = new Hive('/path/to/hive');

// ensureAgent creates memory.md automatically
await hive.ensureAgent({
  id: 'research-assistant-01',
  name: 'Claude',
  role: 'Research Assistant'
});
// File now exists at: /path/to/hive/agents/research-assistant-01/memory.md

```

### Appending Knowledge Durably

```typescript
import { writeFile } from 'fs/promises';
import { join } from 'path';

const memoryPath = join(hive.root, 'agents', agentId, 'memory.md');

// Append with timestamp for chronological organization
const entry = `## ${new Date().toISOString().split('T')[0]} — Key Finding

Discovered that semantic search outperforms keyword matching by 34% 
on long-form memory retrieval tasks.\n\n`;

await writeFile(memoryPath, entry, { flag: 'a' }); // 'a' = append

```

### Querying Across All Agent Memories

```typescript
// From renderer/UI context
const results = await window.cth.get_memory({ 
  query: 'semantic search performance metrics',
  limit: 3 
});

// Returns:
// {
//   source: 'mempalace' | 'grep',
//   matches: [
//     { agent: 'research-assistant-01', excerpt: '...', score: 0.89 },
//     ...
//   ]
// }

```

### Checking System Health

```typescript
const status: MemoryStatus = await window.cth.memoryStatus();

if (!status.palaceReady) {
  console.warn('MemPalace CLI unavailable — falling back to text search');
}

console.log(`Last mining: ${status.lastMineTime?.toLocaleString()}`);
console.log(`${status.totalEmbeddings} embeddings across ${status.indexedAgents.length} agents`);

```

## Architecture Comparison: Design Trade-offs

Munder Difflin's persistent memory design makes intentional trade-offs:

- **Plain text over database**: Maximum portability, universal tool compatibility, trivial backup/restore
- **External CLI over embedded vectors**: Keeps core runtime lightweight; MemPalace can run on GPU-enabled hosts separately
- **Append-only with periodic condensation**: Balances write performance against read efficiency; condensing amortizes cleanup cost
- **Per-agent files over central store**: Natural isolation, easy per-agent export/import, clear ownership boundaries

## Summary

- **Storage**: Each agent's knowledge persists in `HIVE_ROOT/agents/<id>/memory.md` as human-readable markdown
- **Indexing**: The [`MemoryManager`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/memory.ts) mines these files into a MemPalace semantic index located in `<hive>/palace`
- **Maintenance**: The [`Reflector`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/reflect.ts) condenses oversized memories to preserve retrieval performance
- **Access**: The [`get_memory`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/renderer/src/realtime/tools.ts) tool provides semantic search with text fallback; [`hiveMemory`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/preload/index.ts) enables direct file reads
- **IPC**: All memory operations bridge through [[`src/preload/index.ts`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/preload/index.ts)](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/preload/index.ts) for secure renderer-to-main communication

## Frequently Asked Questions

### What happens if the MemPalace CLI is not installed?

Munder Difflin degrades gracefully to **full-text search**. The [`get_memory`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/renderer/src/realtime/tools.ts) tool detects CLI unavailability via [`MemoryStatus.palaceReady`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/memory.ts#L55) and executes a grep-style search across all [`memory.md`](https://github.com/chaitanyagiri/munder-difflin/blob/main/memory.md) files instead. Results lack semantic ranking but remain functionally correct.

### How does memory condensing preserve important information?

The [`Reflector`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/reflect.ts) implements a **three-region structure**: pinned facts (explicitly preserved), condensed summaries (LLM-compressed history), and recent entries (untouched current session). Users can mark critical memories as pinned to exempt them from summarization. The condensing LLM is instructed to preserve entity relationships, factual claims, and action outcomes even when compressing narrative detail.

### Can multiple hives share a single MemPalace index?

No — the palace directory is **hive-scoped**. Each hive instance maintains its own `<hive-home>/palace` to prevent cross-contamination between distinct deployments. However, agents can be migrated between hives by copying their [`memory.md`](https://github.com/chaitanyagiri/munder-difflin/blob/main/memory.md) files; the next mining operation in the destination hive will incorporate the transferred memories.

### Where is the memory file format documented?

The [`memory.md`](https://github.com/chaitanyagiri/munder-difflin/blob/main/memory.md) schema follows **convention over configuration**. Headers use standard markdown (`# Agent Name`, `## Section Title`), chronological entries include ISO date prefixes, and pinned items use a `<!-- PINNED -->` HTML comment marker recognized by the condenser. The [`ensureAgent()`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/hive.ts) implementation in [[`src/main/hive.ts`](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/hive.ts)](https://github.com/chaitanyagiri/munder-difflin/blob/main/src/main/hive.ts) establishes the initial structure for new agents.