# What Are the Two Layers in the ai-memory Storage Model?

> Discover the two layers in the ai-memory storage model: a Wiki layer for source of truth and a SQLite layer for efficient search and indexing. Learn how this dual-architecture works.

- Repository: [Fabio Akita/ai-memory](https://github.com/akitaonrails/ai-memory)
- Tags: internals
- Published: 2026-08-26

---

**The ai-memory storage model implements a dual-layer architecture consisting of a Wiki layer (git-versioned markdown files acting as the source of truth) and a SQLite layer (a derived search index containing FTS5 tables, embeddings, and entity mappings).**

The `akitaonrails/ai-memory` project implements a unique storage architecture designed for AI-assisted knowledge management. Understanding the two layers in the ai-memory storage model is essential for developers who want to leverage its hybrid approach to data persistence, which separates human-readable content from machine-optimized search indexes.

## The Wiki Layer: Git-Versioned Source of Truth

The first layer is the **Wiki layer**, anchored in the `<data_dir>/wiki/` directory. This layer stores all content as plain markdown files, making them directly editable by users and version-controlled by **git2**.

According to the architecture documentation in [[`docs/ARCHITECTURE.md`](https://github.com/akitaonrails/ai-memory/blob/main/docs/ARCHITECTURE.md)](https://github.com/akitaonrails/ai-memory/blob/main/docs/ARCHITECTURE.md), this layer represents the "single source of truth" for the entire system. When you create or modify content, you interact with these markdown files directly. The implementation resides in [[`crates/ai-memory-wiki/src/wiki.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs)](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs), which provides atomic write operations.

```rust
// Access the wiki (source-of-truth) – write a page atomically
use ai_memory_wiki::Wiki;
let wiki = Wiki::open(data_dir.join("wiki"))?;
wiki.write_page("notes/idea.md", "# Idea\nDetails…")?;

```

## The SQLite Layer: High-Performance Derived Index

The second layer is the **SQLite layer**, stored at `<data_dir>/db/memory.sqlite`. Unlike the Wiki layer, this database does not own the primary content. Instead, it maintains a derived index featuring **FTS5** full-text search tables, entity tables, and vector embeddings for fast retrieval.

Located in [[`crates/ai-memory-store/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/lib.rs)](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/lib.rs), this layer provides optimized query capabilities without modifying the underlying markdown source.

```rust
// Query the SQLite index – fast full-text search
use ai_memory_store::Store;
let store = Store::open(data_dir.join("db/memory.sqlite"))?;
let hits = store.query_pages("search term")?;
for hit in hits {
    println!("Found page: {}", hit.title);
}

```

## Architecture Philosophy: Two Layers, One Source of Truth

The relationship between these layers follows a strict hierarchy. The markdown files in the Wiki layer remain the **authoritative source**, while the SQLite database functions as a disposable, rebuildable cache. If the SQLite file becomes corrupted or outdated, the system can regenerate it entirely from the Wiki layer's markdown files.

This design ensures that:

- Your data remains portable (plain text markdown)
- Search operations remain fast (indexed SQLite queries)
- Version control applies only to meaningful content changes, not database indexes

## Summary

- The **ai-memory storage model** separates concerns into two distinct layers: the Wiki layer and the SQLite layer.
- The **Wiki layer** (`<data_dir>/wiki/`) stores markdown files as the git-versioned source of truth, implemented in [`crates/ai-memory-wiki/src/wiki.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs).
- The **SQLite layer** (`<data_dir>/db/memory.sqlite`) contains derived FTS5 indexes and embeddings for fast search, implemented in [`crates/ai-memory-store/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/lib.rs).
- The architecture maintains **"two layers, one source of truth"** as documented in [`docs/ARCHITECTURE.md`](https://github.com/akitaonrails/ai-memory/blob/main/docs/ARCHITECTURE.md), ensuring data integrity while optimizing query performance.

## Frequently Asked Questions

### Which layer serves as the source of truth in ai-memory?

The **Wiki layer** serves as the sole source of truth. The markdown files stored in `<data_dir>/wiki/` are version-controlled and human-editable, while the SQLite layer functions only as a derived index that can be reconstructed from the Wiki content.

### What data structures does the SQLite layer contain?

The SQLite layer contains **FTS5** full-text search tables, entity relationship tables, and vector embeddings. These structures are optimized for fast retrieval queries but do not store the primary markdown content itself.

### Can the SQLite index be regenerated from the Wiki layer?

Yes. Because the SQLite layer is a **derived index** kept in sync with the Wiki layer, it can be deleted and rebuilt entirely from the markdown files in `<data_dir>/wiki/`. This makes the database disposable while preserving all authoritative content in git.

### How do I write to the Wiki layer programmatically?

Use the `ai_memory_wiki::Wiki` struct from [`crates/ai-memory-wiki/src/wiki.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs). The `Wiki::open()` method initializes the connection, and `write_page()` performs atomic writes to specific markdown paths within the wiki directory.