Ai‑memory Query Pipeline Retrieval Methods: Hybrid Search with FTS5, Entities, and Graph Expansion

The memory_query pipeline in akitaonrails/ai-memory combines four independent retrieval streams—FTS5 full‑text search, lexical‑entity matching, graph‑neighbour expansion, and optional vector similarity—fusing their results via Reciprocal‑Rank Fusion (RRF) and an optional LLM reranker.

The akitaonrails/ai-memory repository implements a sophisticated information retrieval system designed to surface relevant context for AI assistants. Written in Rust, the system stores wiki pages and observations in SQLite and exposes a multi‑stage query pipeline through both an MCP (Model Context Protocol) server and a command‑line interface. Understanding how these retrieval methods interact is essential for optimizing search relevance in your own ai‑memory deployments.

Core Retrieval Streams

The query pipeline executes multiple retrieval strategies in parallel, with each stream contributing independent evidence signals.

FTS5 Full‑Text Search

The foundation of the pipeline is a BM25‑style keyword search executed over a SQLite FTS5 index. In crates/ai-memory-store/src/reader.rs, the run_fts5 implementation (around line 472) queries the full‑text index covering both wiki pages and observations. This stream excels at matching exact terminology and phrase patterns, returning an initial candidate set ranked by textual relevance.

Lexical‑Entity Matching

Running concurrently with FTS5, the system performs lexical‑entity matching against normalized tokens stored in the entities table. As documented in crates/ai-memory-mcp/src/server.rs (line 385), this stream identifies class names, function identifiers, or noun phrases that appear as discrete entities in the indexed content. This complements the bag‑of‑words FTS5 approach by recognizing structured references that might be missed by pure text matching.

Graph‑Neighbour Expansion

To capture relational context beyond keyword overlap, the pipeline implements graph‑neighbour expansion. After identifying seed pages via FTS5 and entity matches, the system traverses the wiki link‑graph, promoting pages that are directly linked from the initial hits. This method surfaces conceptually related content that shares no lexical overlap with the query string but occupies a proximal position in the knowledge graph.

Vector (Embedding) Similarity

When an embedder is configured, the pipeline adds vector similarity search as an optional fourth stream. The query string is embedded using the configured provider (e.g., OpenAI), and cosine similarity is computed against stored page embeddings. According to the server implementation at line 1344, this enables "hybrid (FTS5 + entity + vector + graph) query" mode, allowing semantic similarity to influence the final ranking alongside lexical signals.

Result Fusion and Ranking

Independent streams require careful aggregation to produce a coherent final ranking.

Reciprocal‑Rank Fusion (RRF)

Rather than normalizing and averaging raw scores from heterogeneous sources, ai‑memory uses Reciprocal‑Rank Fusion (RRF) to combine the per‑stream rankings. The server code at line 1810 explicitly describes this strategy as "FTS5 + entity‑match + graph RRF + (when configured) vector RRF". RRF assigns a fused score based on the inverse rank of each document across all streams, preventing any single retrieval method from dominating the results while preserving the relative ordering contributions of each stream.

Optional LLM Reranker

Following the RRF fusion stage, the pipeline can apply an LLM‑based reranker for final result refinement. Implemented in crates/ai-memory-llm/src/reranker.rs (line 1), this optional module re‑evaluates the top candidates using a language model to assess semantic relevance. If no reranker is configured, the pipeline terminates after RRF and returns the fused list directly.

Usage Examples

Invoke the retrieval pipeline via the MCP tool (JSON‑RPC):

{
  "jsonrpc": "2.0",
  "method": "tools/call",
  "id": 1,
  "params": {
    "tool_name": "memory_query",
    "args": {
      "query": "rust async runtime",
      "limit": 5,
      "embedder": "openai"
    }
  }
}

Use the CLI wrapper for direct interaction:

# Basic FTS5‑only query

ai-memory query "rust async runtime" --limit 5

# Hybrid query with vector similarity (requires AI_MEMORY_EMBEDDING_PROVIDER)

ai-memory query "rust async runtime" --limit 5 --embedder openai

# Debug individual streams

ai-memory search --fts-only "rust async runtime"
ai-memory search --entity "Runtime"

Key Implementation Files

• crates/ai-memory-mcp/src/server.rs – Orchestrates the memory_query endpoint and documents the retrieval pipeline fusion logic at lines 385, 469, 1344, and 1810.
• crates/ai-memory-store/src/reader.rs – Implements low‑level stream execution including run_fts5, entity lookup, and graph traversal.
• crates/ai-memory-llm/src/embedding.rs – Provides the embedder wrapper that generates query embeddings for the vector similarity stream.
• crates/ai-memory-llm/src/reranker.rs – Contains the optional post‑retrieval LLM reranker.
• crates/ai-memory-cli/src/commands/search.rs – CLI front‑end that constructs and executes the same retrieval requests.

Summary

• Four parallel streams power the pipeline: FTS5 full‑text search, lexical‑entity matching, graph‑neighbour expansion, and optional vector similarity.
• RRF fusion combines stream rankings into a single coherent ordering without requiring score normalization.
• Optional LLM reranking provides final semantic refinement when configured.
• Dual interfaces expose the pipeline via MCP tools and a native CLI, both leveraging the same Rust core in ai-memory-store.

Frequently Asked Questions

Does ai‑memory require vector embeddings to work?

No. The system operates effectively using only FTS5, entity matching, and graph expansion. Vector similarity is an optional stream that activates only when you configure an embedder provider (such as OpenAI) via the embedder parameter or AI_MEMORY_EMBEDDING_PROVIDER environment variable.

How does graph expansion improve search results?

Graph expansion captures relational context that keyword searches miss. When the initial FTS5 and entity streams identify relevant seed pages, the system promotes pages linked from these seeds. This surfaces conceptually adjacent content—such as prerequisite concepts or related implementations—that shares no direct lexical overlap with the query.

What algorithm combines the different retrieval methods?

The pipeline uses Reciprocal‑Rank Fusion (RRF) to merge results. RRF calculates a fused score based on the inverse of each document’s rank within each individual stream, then sums these reciprocals across all streams. This method is robust to different score scales between FTS5, entity, graph, and vector sources.

Can I disable specific retrieval streams?

While the core pipeline always runs FTS5, entity matching, and graph expansion together, you can isolate individual streams for debugging via CLI flags (--fts-only, --entity). The vector stream is opt‑in via the embedder argument, and the LLM reranker is only applied when explicitly configured in your setup.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}