What Are the Two Layers in the ai-memory Storage Model?

The ai-memory storage model implements a dual-layer architecture consisting of a Wiki layer (git-versioned markdown files acting as the source of truth) and a SQLite layer (a derived search index containing FTS5 tables, embeddings, and entity mappings).

The akitaonrails/ai-memory project implements a unique storage architecture designed for AI-assisted knowledge management. Understanding the two layers in the ai-memory storage model is essential for developers who want to leverage its hybrid approach to data persistence, which separates human-readable content from machine-optimized search indexes.

The Wiki Layer: Git-Versioned Source of Truth

The first layer is the Wiki layer, anchored in the <data_dir>/wiki/ directory. This layer stores all content as plain markdown files, making them directly editable by users and version-controlled by git2.

According to the architecture documentation in [docs/ARCHITECTURE.md](https://github.com/akitaonrails/ai-memory/blob/main/docs/ARCHITECTURE.md), this layer represents the "single source of truth" for the entire system. When you create or modify content, you interact with these markdown files directly. The implementation resides in [crates/ai-memory-wiki/src/wiki.rs](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs), which provides atomic write operations.

// Access the wiki (source-of-truth) – write a page atomically
use ai_memory_wiki::Wiki;
let wiki = Wiki::open(data_dir.join("wiki"))?;
wiki.write_page("notes/idea.md", "# Idea\nDetails…")?;

The SQLite Layer: High-Performance Derived Index

The second layer is the SQLite layer, stored at <data_dir>/db/memory.sqlite. Unlike the Wiki layer, this database does not own the primary content. Instead, it maintains a derived index featuring FTS5 full-text search tables, entity tables, and vector embeddings for fast retrieval.

Located in [crates/ai-memory-store/src/lib.rs](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/lib.rs), this layer provides optimized query capabilities without modifying the underlying markdown source.

// Query the SQLite index – fast full-text search
use ai_memory_store::Store;
let store = Store::open(data_dir.join("db/memory.sqlite"))?;
let hits = store.query_pages("search term")?;
for hit in hits {
    println!("Found page: {}", hit.title);
}

Architecture Philosophy: Two Layers, One Source of Truth

The relationship between these layers follows a strict hierarchy. The markdown files in the Wiki layer remain the authoritative source, while the SQLite database functions as a disposable, rebuildable cache. If the SQLite file becomes corrupted or outdated, the system can regenerate it entirely from the Wiki layer's markdown files.

This design ensures that:

• Your data remains portable (plain text markdown)
• Search operations remain fast (indexed SQLite queries)
• Version control applies only to meaningful content changes, not database indexes

Summary

• The ai-memory storage model separates concerns into two distinct layers: the Wiki layer and the SQLite layer.
• The Wiki layer (<data_dir>/wiki/) stores markdown files as the git-versioned source of truth, implemented in crates/ai-memory-wiki/src/wiki.rs.
• The SQLite layer (<data_dir>/db/memory.sqlite) contains derived FTS5 indexes and embeddings for fast search, implemented in crates/ai-memory-store/src/lib.rs.
• The architecture maintains "two layers, one source of truth" as documented in docs/ARCHITECTURE.md, ensuring data integrity while optimizing query performance.

Frequently Asked Questions

Which layer serves as the source of truth in ai-memory?

The Wiki layer serves as the sole source of truth. The markdown files stored in <data_dir>/wiki/ are version-controlled and human-editable, while the SQLite layer functions only as a derived index that can be reconstructed from the Wiki content.

What data structures does the SQLite layer contain?

The SQLite layer contains FTS5 full-text search tables, entity relationship tables, and vector embeddings. These structures are optimized for fast retrieval queries but do not store the primary markdown content itself.

Can the SQLite index be regenerated from the Wiki layer?

Yes. Because the SQLite layer is a derived index kept in sync with the Wiki layer, it can be deleted and rebuilt entirely from the markdown files in <data_dir>/wiki/. This makes the database disposable while preserving all authoritative content in git.

How do I write to the Wiki layer programmatically?

Use the ai_memory_wiki::Wiki struct from crates/ai-memory-wiki/src/wiki.rs. The Wiki::open() method initializes the connection, and write_page() performs atomic writes to specific markdown paths within the wiki directory.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}