# How ai-memory Performs Memory Recall and Search: Architecture and Implementation

> Discover how ai-memory performs memory recall and search using a dual-storage architecture. Learn about Git-backed markdown storage and fast SQLite FTS5 full-text retrieval with optional semantic reranking.

- Repository: [Fabio Akita/ai-memory](https://github.com/akitaonrails/ai-memory)
- Tags: architecture
- Published: 2026-09-09

---

**ai-memory performs memory recall and search by maintaining a dual-storage architecture where coding observations are stored as markdown files in a Git-backed wiki while a derived SQLite FTS5 index enables fast full-text retrieval, with optional semantic reranking via embeddings.**

The **ai-memory** system, developed by akitaonrails, provides persistent memory for AI coding agents through a sophisticated recall pipeline that balances durability with query performance. Understanding how memory recall and search work in this repository requires examining its unique approach to dual storage: human-readable markdown files serve as the source of truth, while a SQLite-based index powers sub-second search without requiring calls to external LLMs.

## The Dual-Storage Architecture (Wiki + SQLite Index)

At the core of ai-memory's recall capability is a write-optimized pipeline that synchronizes human-readable documentation with a machine-optimized search index. This design ensures that every observation captured from coding agents remains accessible both as plain text in a Git repository and as structured data in a relational database.

### Ingestion Pipeline: From Capture to Index

When coding agents generate observations, the system processes them through a lifecycle described in the README as the *capture → consolidate → recall* flow. As detailed in [`README.md`](https://github.com/akitaonrails/ai-memory/blob/main/README.md) lines 61-68, agents first sanitize their observations through lifecycle hooks. When a session terminates, the server consolidates these discrete observations into a single markdown page representing that session's context.

The system then performs an atomic write operation: the wiki writer persists the markdown page to disk while simultaneously updating the search index. In [`crates/ai-memory-store/src/ops.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/ops.rs) lines 76-86, the `upsert_page` function handles this coordination, ensuring that the SQLite index remains consistent with the filesystem state.

### Path Tokenization and FTS5 Indexing

During the upsert operation, the system transforms file paths into searchable tokens. The `path_search_text` function defined in [`crates/ai-memory-store/src/ops.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/ops.rs) lines 15-20 converts page paths into normalized, search-friendly token strings. These tokens are stored in an FTS5 virtual table named `pages_fts`, which provides built-in full-text search capabilities with BM25 ranking.

## Full-Text Search Implementation with FTS5

The search layer leverages SQLite's FTS5 extension to deliver fast, relevance-ranked results without requiring external services. This implementation supports both structured boolean queries and natural language input, making it suitable for direct agent interaction as well as programmatic access.

### Query Sanitization and Natural Language Processing

Raw user queries undergo preprocessing by the `prepare_fts5_query` function located in [`crates/ai-memory-store/src/fts_query.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/fts_query.rs) lines 9-22. This sanitizer performs three critical operations to prevent syntax errors and optimize recall:

- **Operator Detection**: The function checks for explicit FTS5 operators (`OR`, `AND`, `NOT`, `NEAR`, quotation marks, and parentheses). If detected, the query passes through unchanged to preserve user intent.
- **OR-Join Strategy**: For bare natural-language queries, the system splits terms and joins them with `OR` operators instead of the default `AND` behavior. This significantly increases recall; for example, "cross project search strategy" becomes `cross OR project OR search OR strategy`.
- **Token Escaping**: The sanitizer strips stop-words and escapes punctuation marks. Special characters in filenames like `ai-memory` or [`current.md`](https://github.com/akitaonrails/ai-memory/blob/main/current.md) receive proper quoting to prevent SQLite syntax errors, as verified by tests such as `colon_is_not_column_syntax` and `dotted_filename_token_is_quoted`.

### Executing BM25 Ranked Searches

The prepared query executes against the FTS5 virtual table using standard SQL:

```sql
SELECT page_id, rank FROM pages_fts 
WHERE pages_fts MATCH ? 
ORDER BY rank;

```

The `rank` column utilizes BM25 scoring, assigning higher relevance to pages where query terms appear with greater frequency and proximity.

## Recall Pipeline and Result Augmentation

Beyond basic full-text matching, ai-memory enriches search results through entity extraction and graph relationships, with optional semantic similarity scoring for enhanced precision.

### Entity and Link-Based Reranking

Each indexed page contains extracted entities (tags and LLM-generated concepts) stored in the `entities` table. The system maintains relationship data in `entity_page_links`, which connects entities to specific page versions. During the retrieval phase, the pipeline joins these tables to boost results that share entities with the query context, effectively implementing graph-neighbor reranking based on conceptual similarity rather than purely lexical matching.

### Optional Vector Fusion

When configured with an embedding provider, ai-memory extends its retrieval pipeline with semantic search capabilities. Each page maintains a vector representation in the `page_embeddings` table. As noted in [`README.md`](https://github.com/akitaonrails/ai-memory/blob/main/README.md) lines 73-77, the system can blend FTS5 BM25 scores with cosine similarity scores from these embeddings using Reciprocal Rank Fusion (RRF). This hybrid approach combines the precision of keyword matching with the conceptual understanding of vector similarity, though the system functions entirely without embeddings if desired.

## Practical Implementation Examples

Developers can interact with the recall system either programmatically through the Rust API or via the HTTP interface.

### Performing Search via Rust API

To execute a search directly from Rust code using the `ai-memory-store` crate:

```rust
use ai_memory_store::Reader;
use ai_memory_core::{WorkspaceId, ProjectId};

// Establish connection to the SQLite database
let mut conn = rusqlite::Connection::open("/path/to/ai-memory/data/db/memory.sqlite")?;
let reader = Reader::new(&mut conn)?;

// User-provided search query
let raw_query = "how does ai-memory perform memory recall and search";

// Sanitize for FTS5
let fts_query = ai_memory_store::prepare_fts5_query(raw_query);

// Execute search with limit of 10 results
let rows = reader.search_pages(&fts_query, 10)?;

for (page_id, rank) in rows {
    let page = reader.get_page(page_id)?;
    println!("Rank {} – {}", rank, page.title);
}

```

### Querying via HTTP API

The server exposes the search functionality through a REST endpoint:

```bash
curl -G "http://127.0.0.1:49374/api/v1/search" \
     --data-urlencode "q=memory recall" \
     --data "limit=5"

```

The server internally invokes `prepare_fts5_query`, executes the FTS5 query, applies entity enrichment and optional embedding reranking, then returns a JSON array of matching pages.

## Summary

- **ai-memory** implements a dual-storage architecture with markdown files as the source of truth and SQLite FTS5 as the search index.
- The `upsert_page` function in [`crates/ai-memory-store/src/ops.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/ops.rs) atomically synchronizes content between the wiki and the database.
- **FTS5 virtual tables** with BM25 ranking provide fast, relevance-sorted full-text search without LLM dependencies.
- **Query sanitization** via `prepare_fts5_query` (in [`fts_query.rs`](https://github.com/akitaonrails/ai-memory/blob/main/fts_query.rs)) converts natural language to OR-joined tokens while handling special characters safely.
- **Entity augmentation** improves results by considering graph relationships stored in `entities` and `entity_page_links` tables.
- **Optional vector fusion** enables semantic search when embeddings are available, combining cosine similarity with BM25 scores via RRF.

## Frequently Asked Questions

### Does ai-memory require an LLM to perform search?

No. The core search functionality operates entirely within SQLite using FTS5 full-text indexing. As implemented in [`crates/ai-memory-store/src/reader.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/reader.rs), the system executes BM25-ranked queries against the `pages_fts` table without invoking external LLMs. Embeddings and LLM-based entity extraction are optional enhancements, not requirements for basic recall.

### How does ai-memory handle natural language queries?

The system processes natural language through the `prepare_fts5_query` function in [`crates/ai-memory-store/src/fts_query.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/fts_query.rs). For queries without explicit FTS5 operators (like `AND`, `OR`, `NOT`), it splits the input into individual terms and joins them with `OR` operators. This approach maximizes recall by returning pages matching any query term rather than requiring all terms to be present. The function also handles punctuation escaping and stop-word removal to prevent syntax errors.

### What is the relationship between the Git-backed wiki and the SQLite index?

The Git-backed wiki serves as the durable source of truth containing human-readable markdown files. The SQLite database acts as a derived index optimized for query performance. During the consolidation phase, the `upsert_page` operation writes to both stores atomically. If the database is corrupted or deleted, it can be rebuilt from the markdown files, ensuring data durability remains independent of the search infrastructure.

### Can ai-memory perform semantic similarity search?

Yes, when configured with an embedding provider. The system stores vector representations in the `page_embeddings` table and supports cosine similarity calculations. According to the architecture described in [`README.md`](https://github.com/akitaonrails/ai-memory/blob/main/README.md) lines 73-77, the recall pipeline can fuse FTS5 BM25 scores with embedding similarity scores using Reciprocal Rank Fusion (RRF). This hybrid approach allows agents to find conceptually related content even when specific keywords differ.