Understanding the Dual‑Layer Architecture of ai‑memory: SQLite + Markdown Design Explained

The dual‑layer architecture of ai‑memory combines a SQLite data store for structured persistence with a Markdown wiki layer for human‑readable, version‑controlled knowledge representation.

This design enables ai‑memory to deliver both ACID‑guaranteed storage and developer‑friendly editing workflows. The architecture is implemented across two core Rust crates—ai-memory-store and ai-memory-wiki—that remain synchronized through file‑system watchers and atomic write operations.

---

What is the Dual‑Layer Architecture?

The dual‑layer architecture cleanly separates how data is stored from how humans interact with it:

Layer	Technology	Purpose	Guarantees
Data Layer	SQLite with FTS5	Persistent storage of pages, sessions, users, handoffs, and vector embeddings	ACID transactions, full‑text search, vector similarity lookup
Wiki Layer	Markdown files on disk	Human‑readable, editable, Git‑versioned knowledge representation	Atomic file writes, real‑time sync, audit trail

These layers operate bidirectionally. Changes in either layer propagate to the other, ensuring consistency without sacrificing usability.

---

Layer 1: The Persistent Data Store (ai-memory-store)

The data layer lives in the [crates/ai-memory-store/src/lib.rs](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/lib.rs) crate. It provides the primary interface for all structured operations.

Key Components

• Store — The main entry point exposing Reader and Writer interfaces
• Single‑writer actor — Guarantees atomic writes and prevents race conditions
• FTS5 integration — Powers full‑text search via [crates/ai-memory-store/src/fts_query.rs](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/fts_query.rs)
• Schema entities — PagePath, Session, User, Handoff, FTSQuery

Writing to the Store

use ai_memory_store::{Store, PagePath};

let store = Store::open("data/db/memory.sqlite")?;
let page_path = PagePath::new("notes/ai_memory_overview.md")?;
store.write_page(page_path, "## AI‑Memory Overview\nPersistent storage with SQLite.")?;

The Store::open() method initializes the database connection. The write_page() call persists content atomically through the internal writer actor.

---

Layer 2: The Wiki Layer (ai-memory-wiki)

The wiki layer transforms the same knowledge into Markdown files stored on disk. This enables direct editing, Git versioning, and standard Unix tooling.

Key Components

Component	Location	Responsibility
Wiki	[crates/ai-memory-wiki/src/wiki.rs](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs)	Atomic markdown writes, Git commits
Watcher	[crates/ai-memory-wiki/src/watcher.rs](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/watcher.rs)	Filesystem monitoring for wiki → store sync
AtomicFile	Internal module	Temp‑file → rename → fsync pattern for durability

Writing Through the Wiki Layer

use ai_memory_wiki::Wiki;

let wiki = Wiki::new("data/wiki")?;
wiki.write_page(
    "notes/ai_memory_overview.md",
    r#"## AI‑Memory Overview

Persistent storage with SQLite."#
)?;

The write_page() method uses atomic file operations: content writes to a temporary file, renames into place, then calls fsync to ensure durability.

---

Synchronization: Keeping Layers Consistent

The dual‑layer architecture relies on two synchronization paths:

1. Wiki → Store: The Watcher monitors the markdown directory for external changes, parses modified files, and updates the SQLite store.

use ai_memory_wiki::Watcher;

let watcher = Watcher::new("data/wiki")?;
watcher.run(|path, content| {
    // Automatically parsed and synced to store
    store.update_page_from_wiki(path, content);
});

2. Store → Wiki: When the store receives mutations via MCP admin APIs or internal operations, the wiki layer writes corresponding markdown files—preserving full Git history.

This bidirectional flow enables cross‑session continuity and multi‑user collaboration without conflict.

---

Why the Dual‑Layer Architecture Matters

Capability	How the Dual‑Layer Design Delivers
Durability	SQLite provides ACID guarantees and crash recovery
Search performance	FTS5 enables fast full‑text and vector similarity queries
Human editability	Markdown files open in any editor, diff in Git, sync via standard tools
Auditability	Every change leaves a Git commit trail
Auto‑improvement	External tools can modify markdown; watcher propagates changes

The architecture is formally documented in [docs/ARCHITECTURE.md](https://github.com/akitaonrails/ai-memory/blob/main/docs/ARCHITECTURE.md) within the repository.

---

Summary

• Dual‑layer architecture = SQLite data layer + Markdown wiki layer
• ai-memory-store handles persistence, search, and typed queries
• ai-memory-wiki provides human‑readable files, Git integration, and atomic writes
• Bidirectional sync via filesystem watchers keeps both layers consistent
• This design powers ai‑memory's core features: continuity, collaboration, and auto‑improvement

---

Frequently Asked Questions

What problem does the dual‑layer architecture solve?

Traditional databases optimize for machine access but frustrate human workflows. The dual‑layer design gives ai‑memory SQLite's query power alongside Markdown's editability—you get fast searches and files you can open in Vim, commit to Git, and process with standard Unix tools.

How does the wiki layer prevent data loss during writes?

The AtomicFile implementation follows a three‑step pattern: write to temporary file, atomic rename to target path, then fsync to flush buffers. This ensures readers never see partial content and power failures leave the previous file intact.

Can I edit markdown files directly without breaking the system?

Yes. The Watcher continuously monitors the wiki directory. External edits trigger automatic parsing and SQLite updates. The reverse also holds: programmatic store changes write fresh markdown files.

Where is the dual‑layer architecture documented in the codebase?

The design is codified in [docs/ARCHITECTURE.md](https://github.com/akitaonrails/ai-memory/blob/main/docs/ARCHITECTURE.md) and implemented across [crates/ai-memory-store/src/lib.rs](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/lib.rs) (data layer) and [crates/ai-memory-wiki/src/wiki.rs](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs) (wiki layer). The synchronization logic lives in [crates/ai-memory-wiki/src/watcher.rs](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/watcher.rs).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}