# How ai-memory's "Compile, Not Retrieve" Model Works: Technical Architecture Explained

> Discover how ai-memory's "compile, not retrieve" model works. LLMs write knowledge to Git markdown wikis, queries use SQLite for fast, auditable retrieval without LLM calls.

- Repository: [Fabio Akita/ai-memory](https://github.com/akitaonrails/ai-memory)
- Tags: architecture
- Published: 2026-09-11

---

**ai-memory implements a "compile, not retrieve" architecture where LLMs write knowledge into a Git-versioned markdown wiki during consolidation phases, while queries execute against a local SQLite index without calling the LLM, ensuring deterministic, fast, and auditable knowledge retrieval.**

The akitaonrails/ai-memory project redefines how AI agents retain information by treating knowledge as a compiled artifact rather than ephemeral retrieved memory. Instead of querying an LLM for every context lookup, the system uses language models only to compress and structure observations into durable markdown pages. This "compile, not retrieve" approach creates a Karpathy-style LLM wiki that serves as the single source of truth for agent memory.

## The Core Philosophy: Compile Once, Retrieve Many

According to [`docs/ARCHITECTURE.md`](https://github.com/akitaonrails/ai-memory/blob/main/docs/ARCHITECTURE.md), ai-memory treats the knowledge base as "a git-versioned tree of markdown pages on disk that gets *compiled* over time, appended-to."

This design inverts the traditional RAG pattern. Rather than retrieving raw documents and prompting an LLM to synthesize answers for every query, ai-memory **compiles** knowledge upfront. The LLM processes observations into polished wiki pages during consolidation sessions. Subsequent queries retrieve pre-compiled content directly from a structured SQLite index, eliminating per-query LLM latency and cost.

## The Compilation Pipeline

The compilation process transforms transient agent observations into permanent, structured knowledge through a lifecycle-driven pipeline.

### Capture and Persistence

When agents finish sessions, they emit lifecycle-hook events such as `session-end`. The system captures these through `ai_memory_hooks::sanitize`, which validates and structures the incoming data into `WriteCmd` objects.

These commands persist to the `observations` table via a writer actor. This raw capture ensures no data is lost before compilation occurs.

### Summarization and Consolidation

The critical compilation step occurs in [`crates/ai-memory-consolidate/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-consolidate/src/lib.rs). When `AI_MEMORY_LLM_PROVIDER` is configured, the `memory_consolidate` worker processes accumulated observations:

1. **Summarize**: On `session-end`, the server creates an initial summary page (e.g., `sessions/<id>.md`).
2. **Consolidate**: The consolidation pipeline rewrites this page using LLM-generated prose while preserving the original markdown structure.

This bounded worker ensures LLM output is reviewed and structured before becoming source truth.

### Version-Controlled Storage

All compiled knowledge lands in the wiki via [`crates/ai-memory-wiki/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/lib.rs). The `Wiki::write_page` function performs **atomic writes** using tmp-plus-rename patterns followed by Git commits. Every page write is versioned, creating an immutable history where knowledge updates remain traceable.

## The Retrieval Layer: Query Without LLM

Once compiled, knowledge retrieval operates entirely against the SQLite index in [`crates/ai-memory-store/src/pages.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/pages.rs). The `memory_query` tool exposed in [`crates/ai-memory-mcp/src/tools.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-mcp/src/tools.rs) executes three parallel search strategies:

- **FTS5 full-text search** over page titles and bodies.
- **Entity-match** using front-matter `entities` fields.
- **Optional vector-search** when embeddings are enabled.

The system merges these streams using **reciprocal rank fusion (RRF)**, then applies authority multipliers based on tier, pinned status, and tags. No LLM participates in this retrieval flow unless the optional `AI_MEMORY_RERANKER=llm` environment variable is set—and even then, only snippet metadata is sent to the model.

## Benefits of the Compile-Not-Retrieve Architecture

This architectural choice delivers specific technical advantages:

**Deterministic answers** emerge because the wiki is immutable and versioned. Every write through `Wiki::write_page` creates a Git commit, ensuring reproducibility.

**Fast, cheap queries** result from eliminating per-query LLM calls. The SQLite indices (`pages_fts`, `entities`, `page_embeddings`) operate in-process with bounded latency.

**Safety and auditability** come from gating LLM output through the `memory_consolidate` worker. Auto-improve approval workflows can review synthesized content before it enters the permanent wiki.

**Long-term retention** persists beyond LLM provider lifetimes. Pages survive even if `AI_MEMORY_LLM_PROVIDER` is later disabled, while raw observations may be pruned only after successful page compilation.

## Implementation in Code

The following Rust examples demonstrate the compile and retrieve workflow:

```rust
// Compile: Consolidate a session's observations into a wiki page
let consolidate_req = memory_consolidate::Request {
    session_id: Some(session_id),
    multi_page: false,           // single-page rewrite
    ..Default::default()
};
client.memory_consolidate(consolidate_req).await?;

```

```rust
// Retrieve: Query the SQLite index (no LLM involved)
let query_req = memory_query::Request {
    q: "how to reset a project".into(),
    top_k: 5,
    ..Default::default()
};
let hits = client.memory_query(query_req).await?;

```

```rust
// Retrieve: Read full page body when more context is needed
let read_req = memory_read_page::Request {
    path: "procedures/reset-project.md".into(),
    ..Default::default()
};
let page = client.memory_read_page(read_req).await?;
println!("{}", page.body);

```

The first call triggers the **compile** step using the LLM; the subsequent calls perform **retrieve** operations entirely from the local SQLite index.

## Summary

- ai-memory stores knowledge in a **Git-versioned markdown wiki** on disk, making the file system the single source of truth.
- The **consolidation pipeline** in [`crates/ai-memory-consolidate/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-consolidate/src/lib.rs) handles the "compile" phase, using LLMs to synthesize observations into wiki pages.
- Retrieval operates against **SQLite indices** (FTS5, entities, embeddings) via `memory_query`, without invoking the LLM.
- **Atomic Git commits** via `Wiki::write_page` ensure immutable, auditable knowledge history.
- This architecture eliminates per-query LLM costs while providing deterministic, fast, and long-term memory retention.

## Frequently Asked Questions

### What makes ai-memory's approach "compile, not retrieve"?

Traditional RAG systems retrieve raw documents and prompt an LLM to generate answers for every query. ai-memory instead compiles knowledge upfront: the LLM processes observations into structured markdown pages during consolidation. Subsequent queries retrieve these pre-compiled pages directly from SQLite, treating the LLM as a writer rather than a reader.

### How does the compilation process ensure data safety?

The `memory_consolidate` worker runs as a bounded process that can be gated by auto-improve approval workflows. Additionally, `Wiki::write_page` in [`crates/ai-memory-wiki/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/lib.rs) performs atomic tmp-plus-rename writes followed by Git commits, ensuring that compiled knowledge is versioned and immutable before it becomes queryable.

### When does ai-memory use an LLM during retrieval?

By default, retrieval via `memory_query` uses only the SQLite indices (`pages_fts`, `entities`, `page_embeddings`). The LLM is invoked during retrieval only if the `AI_MEMORY_RERANKER=llm` environment variable is enabled, and even then, it processes only the metadata of retrieved snippets rather than the full content.

### What happens to knowledge if the LLM provider is disabled?

Because the wiki is the single source of truth and stored as markdown files on disk, previously compiled pages persist indefinitely even if `AI_MEMORY_LLM_PROVIDER` is removed or changed. Raw observations in the `observations` table may be pruned after successful consolidation, but the compiled wiki pages remain available for retrieval via `memory_query` and `memory_read_page`.