How FTS5, Entity-Match RRF, and Vector Similarity Work Together in ai-memory's memory_query

The memory_query function in akitaonrails/ai-memory performs hybrid retrieval by fusing FTS5 full-text search, entity-match tokenization, graph-neighbor expansion, and optional vector similarity through Reciprocal Rank Fusion (RRF), summing their contributions as 1/(60 + rank) to produce a unified relevance score.

The memory_query tool implements a sophisticated multi-stream retrieval system designed to maximize recall and relevance when searching stored memories. This hybrid approach combines traditional lexical matching with modern semantic search capabilities. Understanding the relationship between FTS5, entity-match RRF, and vector similarity requires examining how these independent ranking streams converge through the Reciprocal Rank Fusion algorithm.

The Four RRF Streams in memory_query

According to crates/ai-memory-store/src/reader.rs (lines 49-52), the hybrid search aggregates four distinct retrieval streams that operate independently before fusion:

• FTS5: SQLite full-text index search for lexical matching against page content
• Entity-Match: Token-based matching against normalized terms extracted from pages and stored in the entity table
• Graph-Neighbour: Link-graph expansion following page connections to include related content
• Vector Similarity: Cosine similarity between embedded queries and stored page embeddings (optional)

Each stream generates its own ranked list of candidate pages. These rankings feed into a unified RRF pipeline where contributions from all active streams are mathematically combined.

FTS5 and Entity-Match RRF Components

These two streams handle lexical matching through complementary mechanisms that capture different aspects of text relevance.

FTS5 Full-Text Search

The FTS5 stream queries the SQLite FTS5 index directly, performing classic full-text search using SQLite's built-in ranking algorithms. As implemented in the hybrid_search function around lines 3600-3655 of reader.rs, this stream produces a rank r₁ based on term frequency, proximity, and other lexical signals inherent to the FTS5 engine.

Entity-Match Tokenization

The entity-match stream operates differently by tokenizing queries into lower-cased, alphanumeric tokens (as documented in comments within reader.rs). These normalized tokens match against the entity table, which stores terms extracted during page indexing. This approach captures exact lexical matches and normalized entities that FTS5's stemming or tokenization rules might handle differently, generating a separate rank r₂.

Vector Similarity Search

When an embedder is configured, the vector similarity stream introduces semantic retrieval capabilities to the hybrid system. The implementation embeds the incoming query (query_vec) and computes cosine similarity against stored page embeddings identified by their (provider, model, dim) tuples.

This stream generates rank r₄ based strictly on semantic distance rather than lexical overlap. As noted in crates/ai-memory-mcp/src/server.rs (line 1810), if no embedder is configured, this stream is omitted entirely and the system falls back to the three-stream version (FTS5 + entity-match + graph).

How RRF Fuses the Streams

The Reciprocal Rank Fusion algorithm combines these independent rankings into a single relevance score. As defined in crates/ai-memory-store/src/reader.rs (lines 63-66), the system uses the canonical RRF constant k = 60.

The fused score calculates as:

fused_score = Σ 1/(60 + rank_i)

For example, a page ranked #1 in FTS5 contributes 1/61 ≈ 0.0164 to the sum, while a page ranked #3 contributes 1/63 ≈ 0.0159. Lower ranks (better positions) contribute more heavily, but signals from all streams are strictly additive. A page that scores well on both entity-match and vector similarity receives a higher fused score than a page that excels on only one stream.

After RRF fusion, PageAuthority::adjust_rank in reader.rs applies a page-authority multiplier to the fused scores. The final list sorts by these adjusted scores and truncates to the requested limit.

Configuring Hybrid Search in Practice

You control which streams participate through the memory_query tool configuration. The following examples demonstrate how to toggle different retrieval mechanisms.

Basic three-stream retrieval (FTS5 + entity-match + graph) without vector search:

let resp = client
    .call_tool("memory_query", json!({
        "query": "how to reset ai-memory",
        "limit": 10
    }))
    .await?;

Enabling the fourth stream by configuring an embedder:

let resp = client
    .call_tool("memory_query", json!({
        "query": "reset ai‑memory workflow",
        "limit": 10,
        "embedder": {
            "provider": "openai",
            "model": "text-embedding-3-small",
            "dim": 1536
        }
    }))
    .await?;

Inspecting individual RRF contributions using the optional reranker:

let resp = client
    .call_tool("memory_query", json!({
        "query": "reset ai‑memory workflow",
        "limit": 5,
        "embedder": { "provider": "openai", "model": "text-embedding-3-small", "dim": 1536 },
        "reranker": "llm"
    }))
    .await?;
// Response includes explain entries showing:
//   "vector": { "rank": 1, "rrf": 0.017 },
//   "fts5": { "rank": 2, "rrf": 0.016 },
//   "entity_match": { "rank": 4, "rrf": 0.012 }

Summary

• RRF unifies four streams: FTS5, entity-match, graph-neighbor, and optional vector similarity each contribute 1/(60 + rank) to the fused score.
• Complementary retrieval: Entity-match captures exact lexical tokens while vector similarity captures semantic meaning; neither stream replaces the other.
• Additive scoring architecture: Pages scoring well on multiple streams receive higher fused scores than single-stream high performers.
• Configurable vector search: Vector similarity requires explicit embedder configuration; otherwise, the system operates as a three-stream hybrid.
• Authority adjustment: RRF scores undergo PageAuthority::adjust_rank multiplication before final sorting and truncation to the requested limit.

Frequently Asked Questions

What is the RRF constant k in ai-memory?

The RRF constant k = 60 is hardcoded in crates/ai-memory-store/src/reader.rs (lines 63-64). This canonical value ensures that lower-ranked (better positioned) results contribute more heavily to the fused score while still allowing signals from higher-ranked items in other streams to influence the final ordering.

Does vector similarity replace the FTS5 and entity-match streams?

No. The vector similarity stream operates in parallel with the lexical streams. According to the source code in reader.rs, these are additive components within the RRF pipeline. While vector similarity captures semantic relationships, entity-match ensures exact lexical tokens are recognized, and FTS5 provides traditional full-text relevance scoring.

How does memory_query handle cases where no embedder is configured?

When no embedder is provided, the vector similarity stream is omitted entirely. As noted in crates/ai-memory-mcp/src/server.rs (line 1810), the system falls back to a three-stream hybrid combining only FTS5, entity-match, and graph-neighbor RRF without semantic search capabilities.

What is the role of PageAuthority in the ranking process?

After RRF fusion calculates the initial fused score, PageAuthority::adjust_rank (implemented in reader.rs) applies a multiplier based on page authority metrics. This adjusts the fused scores before the final sort and truncation to the requested limit, ensuring authoritative pages receive appropriate ranking boosts independent of the retrieval streams' initial rankings.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}