How ai-memory's Hybrid Retrieval System Combines FTS5, Entity-Match, and Graph RRF

ai-memory fuses four independent ranking streams—SQLite FTS5 lexical search, front-matter entity matching, optional vector cosine similarity, and link-graph neighbour expansion—using Reciprocal Rank Fusion (RRF) with a damping constant k=60 to generate unified, transparent relevance scores.

The akitaonrails/ai-memory repository implements a multi-modal search pipeline that unifies lexical, semantic, and structural signals into a single ranking. Its hybrid retrieval system leverages Reciprocal Rank Fusion (RRF) to combine disparate search modalities without requiring score normalization, delivering explainable results across Markdown-based knowledge bases.

The Four Parallel Ranking Streams

The architecture in crates/ai-memory-store/src/reader.rs generates four independent candidate lists. Each stream assigns a 1-based rank to matching documents, which the RRF algorithm later merges.

FTS5 Lexical Search

The primary stream queries the SQLite FTS5 virtual table against Markdown page contents. This stream performs standard full-text boolean matching using the inverted index, returning hits sorted by BM25-derived relevance scores. It handles the bulk of keyword-based retrieval within the corpus.

Entity-Match Extraction

A secondary stream extracts and matches entity names—such as tags, types, and identifiers—from page front-matter. As implemented in crates/ai-memory-wiki/src/wiki.rs, this stream collects exact matches against normalized entity strings, producing a discrete ranking independent of lexical similarity. This captures explicit semantic categorizations that keyword search might overlook.

Optional Vector Cosine Similarity

When an embedder is configured via the memory_query endpoint, the system computes cosine similarity between the query embedding and stored page vectors. This stream is handled by the ai-memory-llm crate and injects dense semantic similarity into the fusion pipeline only when dimensional embeddings are available. If no embedder is attached, the pipeline gracefully degrades to the remaining three streams.

Graph Link-Neighbour Expansion

The fourth stream exploits the internal link graph. Pages linked to high-ranking hits from other streams receive boosted visibility through link-neighbour expansion. The wiki.rs module prepares adjacency data, while reader.rs traverses these relationships to surface structurally relevant neighbours that define the knowledge base topology.

RRF Fusion Algorithm

The system merges stream outputs using Reciprocal Rank Fusion rather than weighted score averaging. This approach naturally accommodates the differing scales of BM25 scores, binary entity matches, and cosine similarities.

The k=60 Fusion Constant

For every document d appearing in stream i, reader.rs computes a contribution using the canonical formula:

contribution = 1.0 / (60 + rank_i(d))

The k = 60 constant is hardcoded in reader.rs to prevent rank dilution and ensure that top-ranked items from any single stream remain competitive against mid-tier results from others.

Score Aggregation Logic

The fused score is the arithmetic sum of contributions across all four streams:

score_rrf(d) = Σ (1 / (60 + rank_i(d)))

Documents missing from a specific stream contribute zero to that stream's summation. The final value represents a unified relevance metric bounded in practice between 0 and 1, though the system preserves raw stream scores alongside the fused total for transparency.

Implementation in reader.rs

The crates/ai-memory-store/src/reader.rs file orchestrates query execution, stream coordination, and result materialization.

Query Parsing and Stream Dispatch

The memory_query entry point parses incoming requests and conditionally attaches an embedder for vector search. The implementation dispatches queries to four independent executors:

1. FTS5 query execution against the content index
2. Entity lookup via front-matter dictionaries
3. Cosine similarity computation (if embedder provided)
4. Graph traversal for neighbour expansion

Each executor returns a ranked list of page identifiers with their respective 1-based ranks.

Result Structuring and Transparency

The final output struct returned to crates/ai-memory-web/src/routes/api.rs includes for each hit:

• The absolute page path
• Raw scores per stream (BM25, entity match boolean, cosine distance, graph depth)
• Individual RRF contributions per stream
• The total fused RRF score
• Authority multipliers applied post-fusion

This granular provenance allows downstream UI components to explain why a specific document surfaced, distinguishing between lexical matches and structural authority.

Optional Post-Processing Pipeline

After RRF fusion, the pipeline may apply additional refinement layers.

LLM Reranker Integration

When configured, crates/ai-memory-llm/src/reranker.rs consumes the top-k fused results and applies a cross-attention reranker to reorder them based on deeper semantic alignment with the query. If no reranker is present, reader.rs returns the RRF scores directly without modification, preserving deterministic retrieval semantics and minimizing latency.

MCP Server Interface

The crates/ai-memory-mcp/src/server.rs module registers the memory_query tool for external consumers via the Model Context Protocol. It handles embedder lifecycle management and streams the fully fused results—complete with per-stream diagnostics—to AI agents and IDE integrations, exposing the complete hybrid retrieval capability through a standardized interface.

Summary

• ai-memory combines FTS5 lexical, entity-match, vector cosine, and graph expansion streams into a unified ranking pipeline.
• Reciprocal Rank Fusion with k=60 merges heterogeneous scores without normalization or calibration.
• The reader.rs implementation provides transparent scoring, exposing per-stream contributions, individual RRF weights, and authority multipliers.
• Optional LLM reranking in reranker.rs can refine results post-fusion when latency constraints permit.
• The system is exposed via the MCP protocol through server.rs, enabling agentic retrieval across local knowledge bases.

Frequently Asked Questions

What is the purpose of using RRF instead of weighted averaging in ai-memory?

Reciprocal Rank Fusion handles the incompatible score distributions across BM25 lexical scores, binary entity matches, and cosine similarities without requiring score calibration. By using rank positions rather than raw magnitudes, RRF ensures that a top-ranked lexical hit contributes competitively against a top-ranked semantic hit, regardless of the underlying numerical scales.

How does the graph expansion stream improve retrieval quality?

The link-neighbour expansion stream in reader.rs boosts pages that are structurally connected to high-ranking results from other streams. This captures relational context that pure content similarity misses, surfacing hub pages and authoritative references that define the knowledge base topology even when they do not contain explicit query keywords.

Can ai-memory operate effectively without vector embeddings?

Yes. The vector cosine similarity stream is entirely optional. When no embedder is configured via the memory_query endpoint, the system defaults to a three-stream hybrid (FTS5, entity-match, and graph expansion), maintaining full retrieval functionality through lexical and structural signals alone.

Where is the fusion constant k=60 defined, and can it be tuned?

The constant k=60 is hardcoded in crates/ai-memory-store/src/reader.rs as part of the core RRF implementation. This value follows canonical RRF literature to balance high-rank prestige against deep-result inclusion. Modification requires source changes to the reader component, as it is not currently exposed as a runtime configuration parameter.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}