How to Set Up Vector Embeddings with sentence-transformers or OpenAI in code-review-graph

The code-review-graph repository provides a flexible embedding subsystem that supports local vector generation via sentence-transformers and remote generation through any OpenAI-compatible API by configuring environment variables and instantiating an EmbeddingStore.

This guide walks through configuring both local and cloud-based embedding providers in code-review-graph. The embedding pipeline is implemented in code_review_graph/embeddings.py, which defines abstract providers, concrete implementations for local and remote backends, and a SQLite-backed storage layer for vector persistence.

Local Embeddings with sentence-transformers

Installation and Setup

Install the optional embeddings extra to pull in sentence-transformers and its dependencies like torch:

pip install "code-review-graph[embeddings]"

Set the CRG_EMBEDDING_MODEL environment variable to select a specific model, or omit it to use the default all-MiniLM-L6-v2:

export CRG_EMBEDDING_MODEL="BAAI/bge-small-en-v1.5"

Pre-warming the Model on Windows

To avoid a deadlock caused by torch importing inside thread pools on Windows, call prewarm_local_embeddings() before any asyncio event loop starts:

from code_review_graph.embeddings import prewarm_local_embeddings

prewarm_local_embeddings()  # Eagerly loads the model on the main thread

This function is defined in code_review_graph/embeddings.py and ensures the model is available in memory before worker threads attempt to access it.

Creating the Embedding Store

Instantiate an EmbeddingStore with a SQLite database path. When provider is omitted or set to "local", the store automatically selects LocalEmbeddingProvider:

from code_review_graph.embeddings import EmbeddingStore

store = EmbeddingStore("path/to/embeddings.db")

The LocalEmbeddingProvider class lazy-loads the sentence-transformers model and maintains a process-wide cache to avoid costly repeated imports, as implemented in the provider's initialization logic.

Embedding Graph Nodes

Embed nodes retrieved from a GraphStore instance:

from code_review_graph.graph import GraphStore

graph = GraphStore("path/to/graph.db")
nodes = [n for f in graph.get_all_files() for n in graph.get_nodes_by_file(f)]

num_embedded = store.embed_nodes(nodes)
print(f"Embedded {num_embedded} nodes")

The EmbeddingStore delegates to LocalEmbeddingProvider.embed(), which converts text to vectors and stores them alongside a hash of the source text to prevent duplicate computations.

Remote Embeddings with OpenAI-Compatible Services

Environment Configuration

Set the required variables to authenticate with any OpenAI-compatible endpoint:

export CRG_OPENAI_API_KEY="sk-..."
export CRG_OPENAI_BASE_URL="https://api.openai.com/v1"  # Or your gateway URL

export CRG_OPENAI_MODEL="text-embedding-3-small"

Optional variables include CRG_OPENAI_DIMENSION to force a specific output dimension and CRG_OPENAI_BATCH_SIZE to override the default batch size of 100.

Suppress the one-time cloud-egress warning printed to stderr by setting:

export CRG_ACCEPT_CLOUD_EMBEDDINGS=1

Without this flag, the library emits a warning via get_provider() when it detects cloud provider configuration.

Provider Selection and Usage

The EmbeddingStore automatically instantiates OpenAIEmbeddingProvider when OpenAI environment variables are present:

store = EmbeddingStore("path/to/embeddings.db")

OpenAIEmbeddingProvider builds a deterministic provider name combining the model and normalized base URL (e.g., openai:text-embedding-3-small@https://api.openai.com/v1), preventing accidental mixing of vectors from different backends.

Embed nodes and perform semantic search identically to the local workflow:

store.embed_nodes(nodes)

results = store.search("async iterator protocol")
for qname, score in results:
    print(f"{qname}: {score:.4f}")

Core Architecture

The embedding subsystem in code_review_graph/embeddings.py consists of several key components:

• EmbeddingProvider (abstract base): Defines the interface with embed(), embed_query(), dimension, and name properties.
• LocalEmbeddingProvider: Implements local inference using sentence-transformers with process-wide caching.
• OpenAIEmbeddingProvider: Implements remote inference via HTTP requests to /v1/embeddings endpoints.
• EmbeddingStore: Wraps SQLite storage, manages vector persistence, and hashes source text to avoid re-embedding unchanged content.
• get_provider(): Factory function that resolves provider names and validates environment variables.

Complete Configuration Examples

Local Embedding Script

from code_review_graph.embeddings import prewarm_local_embeddings, EmbeddingStore
from code_review_graph.graph import GraphStore

# Optional: Pre-warm to prevent Windows deadlock

prewarm_local_embeddings()

graph = GraphStore("path/to/graph.db")
store = EmbeddingStore("path/to/embeddings.db")

nodes = [n for f in graph.get_all_files() for n in graph.get_nodes_by_file(f)]
count = store.embed_nodes(nodes)

print(f"Successfully embedded {count} nodes locally")
for qname, score in store.search("fastapi dependency injection"):
    print(f"  {qname}: {score:.4f}")

OpenAI-Compatible Remote Script

import os
from code_review_graph.embeddings import EmbeddingStore
from code_review_graph.graph import GraphStore

os.environ["CRG_OPENAI_API_KEY"] = "sk-..."
os.environ["CRG_OPENAI_BASE_URL"] = "https://api.openai.com/v1"
os.environ["CRG_OPENAI_MODEL"] = "text-embedding-3-small"
os.environ["CRG_ACCEPT_CLOUD_EMBEDDINGS"] = "1"

graph = GraphStore("path/to/graph.db")
store = EmbeddingStore("path/to/embeddings.db")

store.embed_nodes([n for f in graph.get_all_files() for n in graph.get_nodes_by_file(f)])

for qname, score in store.search("json schema validation"):
    print(f"{qname}: {score:.4f}")

Refreshing Embeddings After Model Changes

To rebuild the vector index when switching models or providers:

from code_review_graph.embeddings import refresh_embeddings
from code_review_graph.graph import GraphStore

graph = GraphStore("graph.db")
stats = refresh_embeddings(graph, provider="openai", model="text-embedding-3-large")
print(stats)  # Output: {'embedded': N, 'purged': M}

Troubleshooting Common Issues

• Windows deadlock: Call prewarm_local_embeddings() before any async operations to prevent torch import deadlocks in worker threads.
• Missing variables: Ensure CRG_OPENAI_API_KEY, CRG_OPENAI_BASE_URL, and CRG_OPENAI_MODEL are all set; get_provider() validates each individually.
• Unexpected re-embedding: Changing the base URL alters the provider identity string, forcing a fresh index; maintain consistent endpoint configurations or explicitly clear the database.
• CI warning spam: Export CRG_ACCEPT_CLOUD_EMBEDDINGS=1 to silence stderr warnings in automated environments.

Summary

• Install local dependencies with pip install "code-review-graph[embeddings]" and set CRG_EMBEDDING_MODEL to configure sentence-transformers.
• Pre-warm local models on Windows using prewarm_local_embeddings() to avoid thread deadlocks.
• Configure remote providers via CRG_OPENAI_API_KEY, CRG_OPENAI_BASE_URL, and CRG_OPENAI_MODEL; suppress warnings with CRG_ACCEPT_CLOUD_EMBEDDINGS=1.
• Instantiate EmbeddingStore with a database path to automatically select the appropriate provider and persist vectors.
• Use embed_nodes() to vectorize graph elements and search() to perform semantic similarity queries against the codebase.

Frequently Asked Questions

How does code-review-graph prevent mixing vectors from different embedding models?

OpenAIEmbeddingProvider generates a deterministic name combining the model and normalized base URL (e.g., openai:text-embedding-3-small@https://api.openai.com/v1), which EmbeddingStore stores alongside each vector. This ensures vectors from different backends or models are never combined in the same search space.

Can I use Azure OpenAI or Ollama instead of the official OpenAI API?

Yes. Set CRG_OPENAI_BASE_URL to your Azure endpoint, LiteLLM gateway, or Ollama server (e.g., http://localhost:11434/v1). The OpenAIEmbeddingProvider works with any service implementing the standard OpenAI /v1/embeddings JSON API format.

Why do I need to call prewarm_local_embeddings() on Windows?

The sentence-transformers library imports torch, which can deadlock when loaded lazily inside a thread pool on Windows. Calling prewarm_local_embeddings() eagerly loads the model on the main thread before any asyncio event loop starts, preventing this race condition.

How do I switch from local embeddings to OpenAI without reprocessing everything?

Use refresh_embeddings() from code_review_graph/embeddings, passing the new provider and model name. This function purges old vectors and regenerates the index using the new configuration, returning statistics about how many nodes were embedded and purged.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}