# How MiniDB Handles Snapshot and WAL Persistence in Kimi Code's Embedded JSON Document Store

> Discover how MiniDB ensures data durability and fast recovery in Kimi Code's embedded JSON store using WAL and snapshot persistence. Learn about its efficient design.

- Repository: [Moonshot AI/kimi-code](https://github.com/MoonshotAI/kimi-code)
- Tags: internals
- Published: 2026-08-16

---

**MiniDB combines an append-only Write-Ahead Log (WAL) for durability with periodic index generation snapshots for fast recovery and compact storage.**

MiniDB powers document persistence in **Kimi Code**, MoonshotAI's AI coding assistant. Understanding its **snapshot and WAL persistence** architecture is essential for developers building reliable embedded applications. This article examines the implementation in `packages/minidb`, tracing how writes become durable and how the system recovers from crashes.

## Write-Ahead Log (WAL) Architecture

Every mutation in MiniDB—`set`, `delete`, or batch operations—flows through an **append-only WAL** before touching the in-memory store.

### WAL Implementation and Configuration

The WAL lives in [`src/wal.ts`](https://github.com/MoonshotAI/kimi-code/blob/main/src/wal.ts) and is exposed to the core database via `this.wal` in `MiniDb`. Writes follow this sequence:

1. `MiniDb.writePath` serializes the operation
2. The in-memory `Store` updates
3. `WAL.append` persists to disk

```typescript
import { MiniDb } from '@moonshot-ai/minidb';

const db = await MiniDb.open({
  dir: './mydb',
  fsyncPolicy: 'everysec',  // flush WAL every second (default)
});

```

The `fsyncPolicy` controls durability trade-offs:

| Policy | Behavior |
|--------|----------|
| `always` | Flush on every write (safest, slowest) |
| `everysec` | Flush once per second (default) |
| `never` | Let OS schedule flushes (fastest, riskiest) |

### WAL Growth and Triggering Snapshots

The WAL grows unbounded until a **generation build threshold** triggers compaction. In [`src/mini-db.ts`](https://github.com/MoonshotAI/kimi-code/blob/main/src/mini-db.ts) (lines 1020–1026), `maybeAutoGenerationBuild` checks:

```typescript
// Simplified from src/mini-db.ts
if (this.generationStale()) {
  // GEN_BUILD_WAL_DELTA_BYTES ≈ 4 MiB threshold exceeded
  this.maintenanceScheduler.schedule(() => this.buildGeneration('auto'));
}

```

Once the WAL delta exceeds **4 MiB**, the background scheduler initiates snapshot creation.

## Snapshot (Index Generation) Persistence

Snapshots in MiniDB are called **index generations**: complete, self-contained materializations of database state.

### Generation Structure and Atomic Publishing

A generation is a directory (`generations/g-NNNNNN/`) containing:

- Binary key-value store image
- Secondary indexes
- DT-column indexes
- Full-text index artifacts

The build process in [`src/generation-builder.ts`](https://github.com/MoonshotAI/kimi-code/blob/main/src/generation-builder.ts) ensures **crash safety** through atomic directory operations:

```

1. Write to temporary directory: g-N.tmp-<random>
2. Rename to final name: g-NNNNNN
3. Update CURRENT symlink
4. Truncate WAL to checkpoint offset

```

This **atomic publish** pattern—documented in [`packages/minidb/AGENTS.md`](https://github.com/MoonshotAI/kimi-code/blob/main/packages/minidb/AGENTS.md) (lines 11–13)—means readers never see partial states, even if the process crashes mid-build.

### Post-Snapshot WAL Truncation

After successful generation publish, [`src/mini-db.ts`](https://github.com/MoonshotAI/kimi-code/blob/main/src/mini-db.ts) truncates the WAL to the checkpoint offset stored in the generation manifest. This keeps the WAL small and recovery fast.

If generation loading fails—corruption, format version mismatch—the system executes **fallback full rebuild**: re-indexing the entire store from scratch using only WAL replay.

## Database Lifecycle and Recovery Flow

The [`src/lifecycle.ts`](https://github.com/MoonshotAI/kimi-code/blob/main/src/lifecycle.ts) module orchestrates open, close, and recovery through `openMiniDb`.

### Opening a Database: Recovery Steps

```typescript
// Called internally by MiniDb.open()
async function openMiniDb(options) {
  // 1. Create/lock database directory
  await acquireDirectoryLock(dir);
  
  // 2. Load latest generation if present
  const generation = await loadLatestGeneration();
  
  // 3. Replay WAL delta to catch up
  await catchUpWalAsync(wal, store, generation?.checkpointOffset);
  
  return new MiniDb(...);
}

```

This three-phase open ensures **fast startup** (via generation) plus **durability** (via WAL replay).

### Read-Only Replica Catch-Up

Read-only instances create isolated scratch directories and replay the WAL without interfering with the writer. From [`src/mini-db.ts`](https://github.com/MoonshotAI/kimi-code/blob/main/src/mini-db.ts) (lines 135–142):

```typescript
const readonlyDb = await MiniDb.open({
  dir: './mydb',
  readOnly: true,  // Creates <dir>.ro-scratch/<pid>-<random>/
});

// Transparently includes all writes flushed to WAL
const doc = await readonlyDb.getAsync('user:123');

```

## Manual Control and Debugging

For testing or operational checkpoints, trigger snapshots explicitly:

```typescript
// Force immediate snapshot generation
await db.buildGeneration('manual');

// Check if generation is stale (exposed for monitoring)
const needsSnapshot = db.generationStale();

```

## Summary

- **WAL persistence** in [`src/wal.ts`](https://github.com/MoonshotAI/kimi-code/blob/main/src/wal.ts) provides append-only durability with configurable `fsyncPolicy` flushing
- **Automatic snapshot triggering** occurs when WAL delta exceeds `GEN_BUILD_WAL_DELTA_BYTES` (≈ 4 MiB), handled in [`src/mini-db.ts`](https://github.com/MoonshotAI/kimi-code/blob/main/src/mini-db.ts)
- **Atomic generation publishing** via [`src/generation-builder.ts`](https://github.com/MoonshotAI/kimi-code/blob/main/src/generation-builder.ts) uses temp directories and symlink swapping for crash safety
- **Recovery** in [`src/lifecycle.ts`](https://github.com/MoonshotAI/kimi-code/blob/main/src/lifecycle.ts) loads the latest generation then replays WAL deltas for consistency
- **Read-only replicas** replay WAL into scratch directories without blocking the writer

## Frequently Asked Questions

### What happens if the WAL grows very large?

The `maybeAutoGenerationBuild` method detects when WAL size exceeds 4 MiB and schedules a background generation build. Once published, the WAL truncates to zero, preventing unbounded growth. If automatic builds are disabled, manual `buildGeneration()` calls or restart recovery will eventually compact the log.

### How does MiniDB prevent data loss during crashes?

The **fsyncPolicy** determines durability guarantees. With `everysec` (default), at most one second of writes may be lost on OS crash; with `always`, every write is durable but slower. The atomic generation publish ensures snapshots are never partial—readers see either the old or new generation, never a corrupted intermediate.

### Can multiple processes open the same MiniDB database?

Only one **writer** can hold the database lock at a time. Multiple **read-only** instances are supported: each creates an isolated `*.ro-scratch/` directory ([`src/mini-db.ts`](https://github.com/MoonshotAI/kimi-code/blob/main/src/mini-db.ts) lines 135–142) and replays WAL deltas independently without interfering with the writer or each other.

### What causes the "fallback full rebuild" path?

Generation loading fails due to checksum mismatch, disk corruption, or an incompatible storage format version. In these cases, [`src/lifecycle.ts`](https://github.com/MoonshotAI/kimi-code/blob/main/src/lifecycle.ts) discards the unusable generation and rebuilds all indexes by replaying the complete WAL from offset zero—slower but guaranteed to produce a valid state.