How code-review-graph Performs Full-Text Search on Code Elements: FTS5, BM25, and Hybrid Ranking Explained

code-review-graph implements full-text search on code elements using SQLite FTS5 virtual tables with BM25 ranking, merged with optional vector embeddings via Reciprocal Rank Fusion.

The tirth8205/code-review-graph repository provides a Python tool that analyzes codebases and builds a queryable graph of code elements. Its search system is designed to locate functions, classes, and types across large repositories efficiently. This article examines how the project implements full-text search on code elements using SQLite's native FTS5 engine, BM25 scoring, and intelligent result boosting.

FTS5 Index Architecture

The foundation of code-review-graph's search capability lies in its dual-table SQLite schema. Raw code elements populate a nodes table, while a mirror FTS5 virtual table named nodes_fts enables full-text operations.

Creating and Rebuilding the FTS Index

The rebuild_fts_index function in code_review_graph/search.py handles index construction. This routine ensures atomic index creation through SQLite transactions, preventing database corruption if the process is interrupted.

from code_review_graph.search import rebuild_fts_index
from code_review_graph.graph import GraphStore

store = GraphStore("my_repo.db")
indexed_count = rebuild_fts_index(store)
print(f"Indexed {indexed_count} code elements")

As implemented in code_review_graph/search.py#L39-L57, the function:

• Drops any existing nodes_fts table
• Recreates it with the FTS5 schema targeting name, qualified name, and signature fields
• Populates the index within a single transaction

This transactional approach guarantees that a crash never leaves the database in a state where the nodes table exists but its search index does not.

Executing BM25 Full-Text Queries

The _fts_search private function executes raw FTS5 queries against the virtual table. According to the source code, this routine performs several critical operations:

1. Escapes double quotes in user input to prevent query syntax errors
2. Executes a MATCH query against nodes_fts
3. Orders results by the BM25 rank column
4. Returns (rowid, -rank) pairs where higher scores indicate better matches

The BM25 algorithm provides relevance-ranked results without requiring manual tf-idf calculations. This is implemented in code_review_graph/search.py#L182-L202.

Hybrid Search: Combining FTS and Vector Embeddings

The public hybrid_search function represents the primary entry point for full-text search on code elements. This function implements a multi-stage fallback pipeline:

Stage	Engine	Purpose
1	_fts_search	Fast BM25 lexical matching
2	Vector embedding search	Semantic similarity (optional)
3	_keyword_search	LIKE pattern fallback

When both FTS and vector search are enabled, results merge through Reciprocal Rank Fusion (rrf_merge). This algorithm, found in code_review_graph/search.py#L151-L166, combines rankings from disparate scoring systems without requiring score normalization.

If all ranked methods return insufficient results, the system falls back to a simple LIKE-based keyword search as shown in code_review_graph/search.py#L256-L286.

Result Boosting and Context-Aware Ranking

Before returning final results, hybrid_search applies multiple heuristic boost strategies to improve relevance:

Kind-Based Detection

The detect_query_kind_boost function (code_review_graph/search.py#L102-L130) infers the user's search intent from query patterns:

• PascalCase queries → boost Class results
• snake_case queries → boost Function results
• UPPER_SNAKE_CASE queries → boost Constant results

Qualified Name Matching

When queries contain dots or extracted identifiers, nodes with matching qualified names receive additional scoring weight. This prioritizes exact namespace matches over partial string occurrences.

Context File Boosting

Callers may supply a context_files list to emphasize results from specific source files. Nodes appearing in these files receive a 1.5× multiplicative boost, useful when the user knows relevant code locations.

from code_review_graph.search import hybrid_search

# Search with contextual file boosting

results = hybrid_search(
    store,
    "calculate_total",
    context_files=["billing.py", "pricing.py"],
    limit=10
)

Result Assembly and Output Format

The final result shaping occurs in code_review_graph/search.py#L442-L469. Selected nodes convert to dictionaries containing:

• Original metadata (kind, file path, line numbers)
• Qualified name and signature
• Normalized score for ranking comparison

Each result includes a score field representing the boosted, normalized ranking value across all search stages.

Complete Working Example

The following demonstration shows the full pipeline from index building to contextual search:

from code_review_graph.graph import GraphStore
from code_review_graph.search import rebuild_fts_index, hybrid_search

# Initialize storage

store = GraphStore("my_repo.db")

# Build FTS index after graph construction

rows_indexed = rebuild_fts_index(store)
print(f"FTS index ready: {rows_indexed} nodes")

# Standard full-text search by name or signature

results = hybrid_search(store, "UserManager", limit=5)
for r in results:
    print(f"{r['kind']} {r['qualified_name']} at {r['file_path']}:{r['line_start']} (score: {r['score']:.3f})")

# Context-boosted search for API-related files

api_results = hybrid_search(
    store,
    "authenticate",
    context_files=["api/routes.py", "middleware/auth.py"],
    limit=3
)

Running this code against a populated graph database returns ranked code elements with BM25 lexical scores, kind-based adjustments, and optional contextual boosting.

Key Implementation Files

Understanding the full-text search on code elements requires examining these source files:

• code_review_graph/search.py — Core FTS5 index creation, BM25 query execution, hybrid merging with RRF, and all boosting heuristics
• code_review_graph/graph.py — GraphStore class providing SQLite connection management and schema
• tests/test_search.py — Documented test cases verifying FTS rebuild, search accuracy, and fallback behavior

Summary

• FTS5 virtual tables mirror the nodes table to enable SQLite-native full-text indexing
• BM25 ranking provides relevance scoring without custom implementation
• Atomic index rebuilds through transactions prevent database corruption
• Hybrid search combines lexical FTS with vector embeddings via Reciprocal Rank Fusion
• Multi-layer boosting uses kind detection, qualified name matching, and context files
• Graceful degradation falls back to LIKE patterns when primary engines fail

Frequently Asked Questions

How does code-review-graph prevent SQL injection in search queries?

The _fts_search function safely escapes double quotes in user input before embedding queries into MATCH expressions. This sanitization prevents FTS5 query syntax manipulation while preserving search intent.

What happens if the FTS index is corrupted or missing?

The rebuild_fts_index function is idempotent and transactional. It can recreate the nodes_fts table at any time from the underlying nodes data. If searches fail entirely, hybrid_search automatically falls back to _keyword_search using standard LIKE patterns.

Why use Reciprocal Rank Fusion instead of score averaging?

RRF merges rankings from FTS BM25 scores and vector cosine similarities without requiring score normalization. Since these scores have different scales and distributions, direct averaging would produce skewed results. RRF treats each engine as a rank-ordering source and produces a robust combined ranking.

Can full-text search work without vector embeddings enabled?

Yes. The hybrid_search function operates in FTS-only mode when embeddings are unavailable or disabled. The vector search stage is conditional, and the system gracefully handles configurations where only lexical search is available.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}