How FTS5 Full-Text Search and Vector Embeddings Work Together in code-review-graph

The code-review-graph repository implements a hybrid search engine that combines BM25-based FTS5 full-text search with semantic vector embeddings using Reciprocal Rank Fusion (RRF), gracefully falling back to keyword search when either subsystem is unavailable.

The code-review-graph project delivers a unified code navigation interface that bridges lexical and semantic retrieval methods. By integrating FTS5 full-text search and vector embeddings, the system matches both exact identifiers and conceptually related symbols, providing robust search capabilities across codebases indexed in SQLite.

The FTS5 Full-Text Search Layer

The repository leverages SQLite's FTS5 extension to provide fast, precise textual matching against code symbols stored in the nodes table.

Building the FTS5 Virtual Table

In code_review_graph/search.py, the rebuild_fts_index function (lines 25-33) manages the creation and maintenance of the nodes_fts virtual table. This function drops the existing virtual table, recreates it, and then populates it with all rows from the underlying nodes table (lines 44-57). According to code_review_graph/migrations.py, this virtual table structure is established in migration v5, providing the foundation for full-text queries against symbol names and signatures.

Executing BM25 Queries

The internal _fts_search function (lines 81-99 in code_review_graph/search.py) executes BM25-ranked queries against nodes_fts. The implementation safely quotes query strings to prevent SQL injection, then returns (node_id, bm25_score) pairs where scores are negated so that larger numbers indicate higher relevance.

from code_review_graph.search import rebuild_fts_index

# Rebuild the FTS index after graph ingestion

fts_rows = rebuild_fts_index(store)  # Returns number of indexed rows

print(f"Indexed {fts_rows} symbols into FTS5")

The Vector Embeddings Layer

While FTS5 handles exact textual matches, semantic similarity search operates through a separate vector storage mechanism that captures conceptual relationships between code elements.

Embedding Store Architecture

Vector embeddings are persisted in a dedicated SQLite table named embeddings, managed by the EmbeddingStore class defined in code_review_graph/embeddings.py. The _embedding_search function in code_review_graph/search.py (lines 12-48) lazily imports this class, validates that the store is available and non-empty, and delegates to emb_store.search(query, limit).

This function returns (qualified_name, similarity_score) pairs, which the system then maps back to corresponding node IDs. If the embedding provider is missing or encounters an error, the function returns an empty list, allowing the hybrid system to continue operating without interruption.

Merging Results with Reciprocal Rank Fusion

When both FTS5 and embedding searches return results, the engine combines them using Reciprocal Rank Fusion (RRF). The rrf_merge function (lines 50-66 in code_review_graph/search.py) assigns each hit a score calculated as 1/(k+rank+1), sums these scores across both result lists, and sorts the merged output by the combined RRF score.

This statistical fusion method balances the high precision of lexical matches with the high recall of semantic similarity, ensuring that exact matches rank highly while semantically relevant results remain discoverable.

Hybrid Search Orchestration

The public hybrid_search function (lines 108-166 in code_review_graph/search.py) orchestrates the entire retrieval pipeline, managing the relationship between FTS5 full-text search and vector embeddings at runtime.

Query Execution Flow

The function first attempts both _fts_search and _embedding_search concurrently. It then classifies the operation mode based on availability (lines 70-79):

• "hybrid": Both FTS5 and embeddings returned results, merged via RRF
• "fts": Only the FTS5 subsystem returned valid results
• "semantic": Only the embedding store returned valid results

Graceful Degradation Strategy

If neither subsystem returns results, the engine executes a fallback keyword-LIKE search. Additional boosting is applied based on node kind, qualified name matches, and context file relevance before the final limited result set is returned.

from code_review_graph.search import hybrid_search

# Pure FTS5 search for exact symbol matches

fts_results = hybrid_search(
    store,
    query="UserService",
    limit=5,
    _out_mode=[]
)

# Pure semantic search for conceptual matches  

semantic_results = hybrid_search(
    store,
    query="authentication flow",
    limit=5,
    _out_mode=[]
)

# True hybrid search combining both modalities

hybrid_results = hybrid_search(
    store,
    query="calculate_total",
    limit=10,
    model="text-embedding-3",
    provider="local"
)

The _out_mode list populates with "fts", "semantic", or "hybrid" to indicate which backends contributed to the specific result set, providing transparency for debugging and performance analysis.

Summary

• Complementary Subsystems: FTS5 provides fast exact matching on textual fields while embeddings deliver semantic similarity for paraphrased or conceptual queries.
• RRF Integration: The rrf_merge function in code_review_graph/search.py statistically fuses ranked results from both modalities using Reciprocal Rank Fusion with the formula 1/(k+rank+1).
• Resilient Architecture: The hybrid_search orchestrator gracefully degrades to keyword-LIKE search when either FTS5 or embedding stores are unavailable, ensuring continuous operation.
• Implementation Files: Core logic resides in code_review_graph/search.py with supporting classes in code_review_graph/embeddings.py, database abstraction in code_review_graph/graph.py, and schema definitions in code_review_graph/migrations.py.

Frequently Asked Questions

What is the relationship between FTS5 full-text search and vector embeddings in code-review-graph?

The two systems operate as complementary retrieval engines within a unified hybrid architecture. FTS5 handles exact lexical matches using BM25 ranking on the nodes_fts virtual table, while vector embeddings provide semantic similarity search through the EmbeddingStore class. The hybrid_search function combines these results using Reciprocal Rank Fusion to produce a single unified ranking that leverages both precision and recall.

How does code-review-graph merge FTS5 and embedding search results?

The repository implements Reciprocal Rank Fusion (RRF) via the rrf_merge function in code_review_graph/search.py (lines 50-66). Each result from the FTS5 and embedding subsystems receives a score of 1/(k+rank+1) based on its position in the respective ranked list. The system sums these scores across both result sets and sorts the final output by the combined RRF score, effectively balancing exact textual matches with semantically similar results.

Can I use only FTS5 or only embeddings in code-review-graph?

Yes. While the default hybrid_search attempts both methods, you can force a specific modality by utilizing the _out_mode parameter or by calling the internal functions directly. The system records the active mode as "fts" when only full-text search succeeds, "semantic" when only embeddings return results, and "hybrid" when both modalities contribute to the final ranked list.

What happens if the embedding provider is unavailable in code-review-graph?

The _embedding_search function gracefully handles missing providers or empty embedding stores by returning an empty list rather than raising exceptions. The hybrid_search orchestrator detects this absence and automatically switches to "fts" mode. If FTS5 also returns no results, the system falls back to a simple keyword-LIKE search, ensuring that search functionality remains operational even when vector services are offline.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}