How to Configure Embeddings with OpenAI-Compatible Endpoints in code-review-graph

Set the CRG_OPENAI_API_KEY and CRG_OPENAI_BASE_URL environment variables to route embedding requests to any OpenAI-compatible service, including Azure OpenAI, LiteLLM, vLLM, LocalAI, or Ollama.

The code-review-graph repository provides a pluggable embedding layer that supports any HTTP service exposing the standard OpenAI /v1/embeddings API. When you configure embeddings with OpenAI-compatible endpoints, the library automatically detects your provider settings through environment variables and handles authentication, batching, and error retries without requiring code changes.

Environment Variable Configuration

The embedding system recognizes four primary environment variables to establish the connection. According to the source code in code_review_graph/embeddings.py, you must define:

• CRG_OPENAI_API_KEY – The authentication secret for your endpoint
• CRG_OPENAI_BASE_URL – The root URL of the compatible service (e.g., https://api.openai.com/v1, http://localhost:5000/v1, or your Azure OpenAI deployment URL)
• CRG_OPENAI_MODEL – The model identifier string (e.g., text-embedding-3-large, text-embedding-3-small)
• CRG_OPENAI_DIMENSION – Optional override for the vector output size

You may also tune performance with CRG_OPENAI_BATCH_SIZE to control how many texts are sent per request (default is 100).

export CRG_OPENAI_API_KEY="sk-xxxxxxxxxxxxxxxxxxxx"
export CRG_OPENAI_BASE_URL="http://localhost:5000/v1"
export CRG_OPENAI_MODEL="text-embedding-3-small"
export CRG_OPENAI_BATCH_SIZE=50

Provider Selection Mechanism

The get_provider() function (lines 66-78 in code_review_graph/embeddings.py) inspects the runtime environment to determine which backend to instantiate. When both CRG_OPENAI_API_KEY and CRG_OPENAI_BASE_URL are present, the function automatically selects the openai provider, passing the resolved configuration to the OpenAIEmbeddingProvider class.

This design allows seamless switching between local development (using Ollama or LocalAI) and production deployments (using Azure OpenAI or the official OpenAI API) without modifying application code.

Safety Checks for Cloud Egress

The implementation includes defensive checks to prevent accidental data leakage to third-party clouds. Two key functions handle this:

• _is_localhost_url() (lines 126-138) – Detects if the configured CRG_OPENAI_BASE_URL points to localhost, 127.0.0.1, or local Unix sockets
• _warn_cloud_egress() (lines 140-164) – Prints a one-time warning to stderr when the endpoint appears to be a remote cloud service

To suppress the cloud egress warning for CI pipelines or trusted environments, set CRG_ACCEPT_CLOUD_EMBEDDINGS=1.

export CRG_ACCEPT_CLOUD_EMBEDDINGS=1

Provider Identity and Caching

When constructing the provider, the _make_host_key() function (lines 100-120) normalizes the host portion of the base URL to create a unique provider identity. This mechanism ensures that changing the hostname or path creates a distinct provider instance, preventing accidental mixing of vectors from different backends in the SQLite cache.

For example, switching from http://localhost:5000/v1 to http://localhost:6000/v1 generates a new provider identity, triggering a separate embedding namespace in the local cache.

Making Embedding Requests

The OpenAIEmbeddingProvider.embed() method batches input texts according to CRG_OPENAI_BATCH_SIZE and delegates to _call_api() (lines 390-560), which performs an HTTP POST to <base_url>/embeddings with the following behavior:

• Retries transient network errors automatically
• Validates response structure and index handling
• Discovers embedding dimensions dynamically if not forced via CRG_OPENAI_DIMENSION
• Reports detailed error messages for authentication or rate limit failures

Practical Usage Examples

CLI Usage

After setting the environment variables, all CLI commands automatically use the configured endpoint:

crg search "graph traversal algorithm"
crg embed --provider openai --model "text-embedding-3-small"

Library Usage

Use the embedding store directly in Python applications:

from code_review_graph.embeddings import EmbeddingStore, embed_all_nodes

# Automatically picks the OpenAI provider from environment variables

store = EmbeddingStore(db_path="embeddings.db")
embedded = embed_all_nodes(graph, store)
print(f"Embedded {embedded} nodes")

Refreshing Embeddings When Switching Models

When you change models or endpoints, use the refresh utility to update cached vectors:

from code_review_graph.embeddings import refresh_embeddings

result = refresh_embeddings(
    graph_store=graph,
    provider="openai",
    model="text-embedding-3-large",
)
print(result)  # {'embedded': <n>, 'purged': <m>}

Summary

• Configure embeddings with OpenAI-compatible endpoints by setting CRG_OPENAI_API_KEY and CRG_OPENAI_BASE_URL before running the CLI or importing the library
• The get_provider() function in code_review_graph/embeddings.py auto-detects OpenAI-compatible configurations at lines 66-78
• Set CRG_ACCEPT_CLOUD_EMBEDDINGS=1 to suppress egress warnings for remote endpoints
• The provider identity system prevents cache collisions when switching between different base URLs or models
• Batch size defaults to 100 but is configurable via CRG_OPENAI_BATCH_SIZE

Frequently Asked Questions

What environment variables are required to configure OpenAI-compatible endpoints?

You must define CRG_OPENAI_API_KEY and CRG_OPENAI_BASE_URL for the provider to activate. Optionally specify CRG_OPENAI_MODEL (default varies by implementation), CRG_OPENAI_DIMENSION for fixed vector sizes, and CRG_OPENAI_BATCH_SIZE to tune request volume (default 100).

How do I disable the cloud egress warning?

Set CRG_ACCEPT_CLOUD_EMBEDDINGS=1 in your environment. This bypasses the _warn_cloud_egress() check defined at lines 140-164 in code_review_graph/embeddings.py, which normally alerts you when sending data to non-localhost URLs.

Can I use local embedding servers like Ollama or LocalAI?

Yes. Point CRG_OPENAI_BASE_URL to your local endpoint (e.g., http://localhost:11434/v1 for Ollama). The _is_localhost_url() function (lines 126-138) detects local addresses and skips the cloud egress warning automatically.

Why does changing the base URL create a new provider identity?

The _make_host_key() function (lines 100-120) hashes the normalized host and path of the base URL to generate a cache key. This ensures that vectors from different backends (e.g., Azure OpenAI vs. local LiteLLM) remain isolated in the SQLite cache, preventing semantic mixing of incompatible embedding spaces.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}