How Semantic Search Integrates with Sentence-Transformers in Code-Review-Graph

Code-Review-Graph implements semantic search by combining BM25 full-text search with vector embeddings from the sentence-transformers library, using a hybrid approach with Reciprocal Rank Fusion to rank results by both lexical and semantic similarity.

The tirth8205/code-review-graph repository provides a production-ready semantic search pipeline that leverages the sentence-transformers library to enable natural language querying of codebases. By integrating dense vector retrieval with traditional FTS5 BM25 search, the system delivers accurate code discovery even when queries don't match exact keywords, while maintaining graceful fallbacks for environments without PyTorch.

The Hybrid Search Architecture

The system implements a hybrid search strategy that merges two retrieval paradigms. Traditional full-text search using SQLite FTS5 BM25 handles exact keyword matching, while neural embeddings capture conceptual similarity between queries and code elements. The hybrid_search() function in code_review_graph/search.py orchestrates this merger, receiving queries and forwarding them to the embedding provider for vector generation before fusing results.

This dual approach ensures that searching for "authentication token validation" returns not only functions containing those exact words, but also related implementations dealing with "JWT verification" or "OAuth credential checking" through semantic proximity.

Setting Up the Embedding Provider

Lazy Loading with LocalEmbeddingProvider

The LocalEmbeddingProvider class in code_review_graph/embeddings.py serves as the default local embedding implementation. When a semantic query executes, the provider lazily loads a SentenceTransformer model (defaulting to all-MiniLM-L6-v2) and caches it process-wide. This caching mechanism prevents repeated imports that can cause deadlocks on Windows threaded environments, as implemented in lines 39-48.

# Example – Run a semantic search with the default local model

from code_review_graph.tools.query import semantic_search_nodes

result = semantic_search_nodes(
    query="authentication token validation",
    kind="Function",
    limit=10,
    detail_level="standard",
)

print(result["summary"])
print(result["search_mode"])   # → "semantic"

print(result["results"][0]["name"])

Model Pre-warming for Thread Safety

To avoid heavy PyTorch initialization within async worker pools, the codebase provides prewarm_local_embeddings() in code_review_graph/embeddings.py (lines 85-92). Calling this function from the main thread at server startup eagerly loads the model, ensuring that torch initialization happens outside the asyncio event loop.

# Pre-warm the embedding model at startup (critical for Windows)

from code_review_graph.embeddings import prewarm_local_embeddings

prewarm_local_embeddings()   # Loads SentenceTransformer before asyncio loops

The Query Execution Pipeline

From Query to Vector in semantic_search_nodes

The public tool semantic_search_nodes() defined in code_review_graph/tools/query.py (lines 997-1000) serves as the primary entry point for semantic queries. This function invokes hybrid_search(), which coordinates between the embedding provider and the FTS5 index. The tool reports the search mode (semantic, keyword, or fts) based on which component contributed the most results, and supports both minimal and full payload outputs.

Reciprocal Rank Fusion Implementation

The core retrieval logic uses Reciprocal Rank Fusion to merge BM25 scores with cosine similarity scores from the sentence-transformers embeddings. This technique ensures that results ranking highly in either lexical or semantic relevance surface in the final output, preventing pure keyword results from overwhelming semantically relevant matches, or vice versa.

Fallbacks and Resilience

When the embedding provider is unavailable—such as when sentence-transformers is not installed—the system gracefully degrades to pure keyword/FTS search without failing. This behavior is handled in code_review_graph/tools/query.py (lines 78-81), ensuring that search functionality remains available even in resource-constrained environments.

# Switch to a cloud provider without installing sentence-transformers

import os
os.environ["CRG_EMBEDDING_PROVIDER"] = "openai"
os.environ["OPENAI_API_KEY"] = "sk-…"

from code_review_graph.tools.query import semantic_search_nodes
result = semantic_search_nodes("database connection pooling")
print(result["search_mode"])   # Uses OpenAI embeddings instead

Summary

• Hybrid retrieval: The system combines FTS5 BM25 full-text search with sentence-transformers embeddings using Reciprocal Rank Fusion in hybrid_search().
• Lazy loading: LocalEmbeddingProvider caches the all-MiniLM-L6-v2 model process-wide to prevent thread deadlocks on Windows.
• Pre-warming: Call prewarm_local_embeddings() from the main thread before starting async workers to avoid PyTorch initialization delays.
• Graceful degradation: Search falls back to keyword/FTS mode when sentence-transformers is unavailable, as handled in code_review_graph/tools/query.py.
• Pluggable providers: Set CRG_EMBEDDING_PROVIDER environment variable to switch between local sentence-transformers models and cloud APIs like OpenAI.

Frequently Asked Questions

What embedding model does Code-Review-Graph use by default?

The LocalEmbeddingProvider defaults to the all-MiniLM-L6-v2 model from the sentence-transformers library. This model provides a strong balance between embedding quality and inference speed, generating 384-dimensional vectors that capture semantic meaning while remaining lightweight enough for local CPU inference.

How does the system handle Windows threading issues with PyTorch?

The implementation in code_review_graph/embeddings.py uses process-wide model caching to prevent repeated imports of sentence-transformers that can trigger deadlocks on Windows threaded environments. Additionally, the prewarm_local_embeddings() function allows eager loading from the main thread, ensuring that the heavy torch initialization occurs outside any asyncio worker pools where it might conflict with Windows multithreading constraints.

Can I use semantic search without installing sentence-transformers?

Yes. If the embedding provider is unavailable, semantic_search_nodes() gracefully falls back to pure keyword and FTS5 search, as implemented in lines 78-81 of code_review_graph/tools/query.py. Alternatively, you can configure external providers like OpenAI by setting the CRG_EMBEDDING_PROVIDER environment variable, bypassing the need for local PyTorch and sentence-transformers installations entirely.

What is Reciprocal Rank Fusion and why is it used?

Reciprocal Rank Fusion (RRF) is a rank aggregation algorithm that combines result lists from different retrieval methods without requiring score normalization. In code_review_graph/search.py, RRF merges the ranked outputs from BM25 lexical search and sentence-transformers vector similarity, ensuring that items ranking highly in either modality receive appropriate final rankings. This approach balances exact keyword matching with conceptual semantic understanding better than simple score averaging would allow.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}