# Understanding the Dual‑Layer Architecture of ai‑memory: SQLite + Markdown Design Explained

> Discover ai-memory's dual-layer architecture. It uses SQLite for structured data and Markdown for human-readable, version-controlled knowledge. Learn more!

- Repository: [Fabio Akita/ai-memory](https://github.com/akitaonrails/ai-memory)
- Tags: architecture
- Published: 2026-09-01

---

**The dual‑layer architecture of ai‑memory combines a SQLite data store for structured persistence with a Markdown wiki layer for human‑readable, version‑controlled knowledge representation.**

This design enables ai‑memory to deliver both ACID‑guaranteed storage and developer‑friendly editing workflows. The architecture is implemented across two core Rust crates—**`ai-memory-store`** and **`ai-memory-wiki`**—that remain synchronized through file‑system watchers and atomic write operations.

---

## What is the Dual‑Layer Architecture?

The dual‑layer architecture cleanly separates **how data is stored** from **how humans interact with it**:

| Layer | Technology | Purpose | Guarantees |
|-------|-----------|---------|------------|
| **Data Layer** | SQLite with FTS5 | Persistent storage of pages, sessions, users, handoffs, and vector embeddings | ACID transactions, full‑text search, vector similarity lookup |
| **Wiki Layer** | Markdown files on disk | Human‑readable, editable, Git‑versioned knowledge representation | Atomic file writes, real‑time sync, audit trail |

These layers operate bidirectionally. Changes in either layer propagate to the other, ensuring consistency without sacrificing usability.

---

## Layer 1: The Persistent Data Store (`ai-memory-store`)

The data layer lives in the [[`crates/ai-memory-store/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/lib.rs)](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/lib.rs) crate. It provides the primary interface for all structured operations.

### Key Components

- **`Store`** — The main entry point exposing `Reader` and `Writer` interfaces
- **Single‑writer actor** — Guarantees atomic writes and prevents race conditions
- **FTS5 integration** — Powers full‑text search via [[`crates/ai-memory-store/src/fts_query.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/fts_query.rs)](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/fts_query.rs)
- **Schema entities** — `PagePath`, `Session`, `User`, `Handoff`, `FTSQuery`

### Writing to the Store

```rust
use ai_memory_store::{Store, PagePath};

let store = Store::open("data/db/memory.sqlite")?;
let page_path = PagePath::new("notes/ai_memory_overview.md")?;
store.write_page(page_path, "## AI‑Memory Overview\nPersistent storage with SQLite.")?;

```

The `Store::open()` method initializes the database connection. The `write_page()` call persists content atomically through the internal writer actor.

---

## Layer 2: The Wiki Layer (`ai-memory-wiki`)

The wiki layer transforms the same knowledge into **Markdown files** stored on disk. This enables direct editing, Git versioning, and standard Unix tooling.

### Key Components

| Component | Location | Responsibility |
|-----------|----------|----------------|
| **`Wiki`** | [[`crates/ai-memory-wiki/src/wiki.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs)](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs) | Atomic markdown writes, Git commits |
| **`Watcher`** | [[`crates/ai-memory-wiki/src/watcher.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/watcher.rs)](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/watcher.rs) | Filesystem monitoring for wiki → store sync |
| **`AtomicFile`** | Internal module | Temp‑file → rename → fsync pattern for durability |

### Writing Through the Wiki Layer

```rust
use ai_memory_wiki::Wiki;

let wiki = Wiki::new("data/wiki")?;
wiki.write_page(
    "notes/ai_memory_overview.md",
    r#"## AI‑Memory Overview

Persistent storage with SQLite."#
)?;

```

The `write_page()` method uses atomic file operations: content writes to a temporary file, renames into place, then calls `fsync` to ensure durability.

---

## Synchronization: Keeping Layers Consistent

The dual‑layer architecture relies on two synchronization paths:

1. **Wiki → Store**: The `Watcher` monitors the markdown directory for external changes, parses modified files, and updates the SQLite store.

```rust
use ai_memory_wiki::Watcher;

let watcher = Watcher::new("data/wiki")?;
watcher.run(|path, content| {
    // Automatically parsed and synced to store
    store.update_page_from_wiki(path, content);
});

```

2. **Store → Wiki**: When the store receives mutations via MCP admin APIs or internal operations, the wiki layer writes corresponding markdown files—preserving full Git history.

This bidirectional flow enables **cross‑session continuity** and **multi‑user collaboration** without conflict.

---

## Why the Dual‑Layer Architecture Matters

| Capability | How the Dual‑Layer Design Delivers |
|------------|-----------------------------------|
| **Durability** | SQLite provides ACID guarantees and crash recovery |
| **Search performance** | FTS5 enables fast full‑text and vector similarity queries |
| **Human editability** | Markdown files open in any editor, diff in Git, sync via standard tools |
| **Auditability** | Every change leaves a Git commit trail |
| **Auto‑improvement** | External tools can modify markdown; watcher propagates changes |

The architecture is formally documented in [[`docs/ARCHITECTURE.md`](https://github.com/akitaonrails/ai-memory/blob/main/docs/ARCHITECTURE.md)](https://github.com/akitaonrails/ai-memory/blob/main/docs/ARCHITECTURE.md) within the repository.

---

## Summary

- **Dual‑layer architecture** = SQLite data layer + Markdown wiki layer
- **`ai-memory-store`** handles persistence, search, and typed queries
- **`ai-memory-wiki`** provides human‑readable files, Git integration, and atomic writes
- **Bidirectional sync** via filesystem watchers keeps both layers consistent
- This design powers ai‑memory's core features: continuity, collaboration, and auto‑improvement

---

## Frequently Asked Questions

### What problem does the dual‑layer architecture solve?

Traditional databases optimize for machine access but frustrate human workflows. The dual‑layer design gives ai‑memory **SQLite's query power** alongside **Markdown's editability**—you get fast searches *and* files you can open in Vim, commit to Git, and process with standard Unix tools.

### How does the wiki layer prevent data loss during writes?

The [`AtomicFile`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs) implementation follows a three‑step pattern: write to temporary file, atomic rename to target path, then `fsync` to flush buffers. This ensures readers never see partial content and power failures leave the previous file intact.

### Can I edit markdown files directly without breaking the system?

Yes. The [`Watcher`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/watcher.rs) continuously monitors the wiki directory. External edits trigger automatic parsing and SQLite updates. The reverse also holds: programmatic store changes write fresh markdown files.

### Where is the dual‑layer architecture documented in the codebase?

The design is codified in [[`docs/ARCHITECTURE.md`](https://github.com/akitaonrails/ai-memory/blob/main/docs/ARCHITECTURE.md)](https://github.com/akitaonrails/ai-memory/blob/main/docs/ARCHITECTURE.md) and implemented across [[`crates/ai-memory-store/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/lib.rs)](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/lib.rs) (data layer) and [[`crates/ai-memory-wiki/src/wiki.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs)](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs) (wiki layer). The synchronization logic lives in [[`crates/ai-memory-wiki/src/watcher.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/watcher.rs)](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/watcher.rs).