# How to Configure Embeddings with OpenAI-Compatible Endpoints in code-review-graph

> Learn to configure embeddings with OpenAI-compatible endpoints in code-review-graph. Set environment variables CRG_OPENAI_API_KEY and CRG_OPENAI_BASE_URL to integrate with Azure OpenAI, LiteLLM, vLLM, LocalAI, and Ollama.

- Repository: [Tirth Kanani/code-review-graph](https://github.com/tirth8205/code-review-graph)
- Tags: how-to-guide
- Published: 2026-07-25

---

**Set the `CRG_OPENAI_API_KEY` and `CRG_OPENAI_BASE_URL` environment variables to route embedding requests to any OpenAI-compatible service, including Azure OpenAI, LiteLLM, vLLM, LocalAI, or Ollama.**

The `code-review-graph` repository provides a pluggable embedding layer that supports any HTTP service exposing the standard OpenAI `/v1/embeddings` API. When you configure embeddings with OpenAI-compatible endpoints, the library automatically detects your provider settings through environment variables and handles authentication, batching, and error retries without requiring code changes.

## Environment Variable Configuration

The embedding system recognizes four primary environment variables to establish the connection. According to the source code in [`code_review_graph/embeddings.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/embeddings.py), you must define:

- **`CRG_OPENAI_API_KEY`** – The authentication secret for your endpoint
- **`CRG_OPENAI_BASE_URL`** – The root URL of the compatible service (e.g., `https://api.openai.com/v1`, `http://localhost:5000/v1`, or your Azure OpenAI deployment URL)
- **`CRG_OPENAI_MODEL`** – The model identifier string (e.g., `text-embedding-3-large`, `text-embedding-3-small`)
- **`CRG_OPENAI_DIMENSION`** – Optional override for the vector output size

You may also tune performance with **`CRG_OPENAI_BATCH_SIZE`** to control how many texts are sent per request (default is 100).

```bash
export CRG_OPENAI_API_KEY="sk-xxxxxxxxxxxxxxxxxxxx"
export CRG_OPENAI_BASE_URL="http://localhost:5000/v1"
export CRG_OPENAI_MODEL="text-embedding-3-small"
export CRG_OPENAI_BATCH_SIZE=50

```

## Provider Selection Mechanism

The `get_provider()` function (lines 66-78 in [`code_review_graph/embeddings.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/embeddings.py)) inspects the runtime environment to determine which backend to instantiate. When both `CRG_OPENAI_API_KEY` and `CRG_OPENAI_BASE_URL` are present, the function automatically selects the **openai** provider, passing the resolved configuration to the `OpenAIEmbeddingProvider` class.

This design allows seamless switching between local development (using Ollama or LocalAI) and production deployments (using Azure OpenAI or the official OpenAI API) without modifying application code.

## Safety Checks for Cloud Egress

The implementation includes defensive checks to prevent accidental data leakage to third-party clouds. Two key functions handle this:

- **`_is_localhost_url()`** (lines 126-138) – Detects if the configured `CRG_OPENAI_BASE_URL` points to localhost, 127.0.0.1, or local Unix sockets
- **`_warn_cloud_egress()`** (lines 140-164) – Prints a one-time warning to stderr when the endpoint appears to be a remote cloud service

To suppress the cloud egress warning for CI pipelines or trusted environments, set **`CRG_ACCEPT_CLOUD_EMBEDDINGS=1`**.

```bash
export CRG_ACCEPT_CLOUD_EMBEDDINGS=1

```

## Provider Identity and Caching

When constructing the provider, the `_make_host_key()` function (lines 100-120) normalizes the host portion of the base URL to create a unique provider identity. This mechanism ensures that changing the hostname or path creates a distinct provider instance, preventing accidental mixing of vectors from different backends in the SQLite cache.

For example, switching from `http://localhost:5000/v1` to `http://localhost:6000/v1` generates a new provider identity, triggering a separate embedding namespace in the local cache.

## Making Embedding Requests

The `OpenAIEmbeddingProvider.embed()` method batches input texts according to `CRG_OPENAI_BATCH_SIZE` and delegates to `_call_api()` (lines 390-560), which performs an HTTP POST to `<base_url>/embeddings` with the following behavior:

- Retries transient network errors automatically
- Validates response structure and index handling
- Discovers embedding dimensions dynamically if not forced via `CRG_OPENAI_DIMENSION`
- Reports detailed error messages for authentication or rate limit failures

## Practical Usage Examples

### CLI Usage

After setting the environment variables, all CLI commands automatically use the configured endpoint:

```bash
crg search "graph traversal algorithm"
crg embed --provider openai --model "text-embedding-3-small"

```

### Library Usage

Use the embedding store directly in Python applications:

```python
from code_review_graph.embeddings import EmbeddingStore, embed_all_nodes

# Automatically picks the OpenAI provider from environment variables

store = EmbeddingStore(db_path="embeddings.db")
embedded = embed_all_nodes(graph, store)
print(f"Embedded {embedded} nodes")

```

### Refreshing Embeddings When Switching Models

When you change models or endpoints, use the refresh utility to update cached vectors:

```python
from code_review_graph.embeddings import refresh_embeddings

result = refresh_embeddings(
    graph_store=graph,
    provider="openai",
    model="text-embedding-3-large",
)
print(result)  # {'embedded': <n>, 'purged': <m>}

```

## Summary

- **Configure embeddings with OpenAI-compatible endpoints** by setting `CRG_OPENAI_API_KEY` and `CRG_OPENAI_BASE_URL` before running the CLI or importing the library
- The `get_provider()` function in [`code_review_graph/embeddings.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/embeddings.py) auto-detects OpenAI-compatible configurations at lines 66-78
- Set `CRG_ACCEPT_CLOUD_EMBEDDINGS=1` to suppress egress warnings for remote endpoints
- The provider identity system prevents cache collisions when switching between different base URLs or models
- Batch size defaults to 100 but is configurable via `CRG_OPENAI_BATCH_SIZE`

## Frequently Asked Questions

### What environment variables are required to configure OpenAI-compatible endpoints?

You must define `CRG_OPENAI_API_KEY` and `CRG_OPENAI_BASE_URL` for the provider to activate. Optionally specify `CRG_OPENAI_MODEL` (default varies by implementation), `CRG_OPENAI_DIMENSION` for fixed vector sizes, and `CRG_OPENAI_BATCH_SIZE` to tune request volume (default 100).

### How do I disable the cloud egress warning?

Set `CRG_ACCEPT_CLOUD_EMBEDDINGS=1` in your environment. This bypasses the `_warn_cloud_egress()` check defined at lines 140-164 in [`code_review_graph/embeddings.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/embeddings.py), which normally alerts you when sending data to non-localhost URLs.

### Can I use local embedding servers like Ollama or LocalAI?

Yes. Point `CRG_OPENAI_BASE_URL` to your local endpoint (e.g., `http://localhost:11434/v1` for Ollama). The `_is_localhost_url()` function (lines 126-138) detects local addresses and skips the cloud egress warning automatically.

### Why does changing the base URL create a new provider identity?

The `_make_host_key()` function (lines 100-120) hashes the normalized host and path of the base URL to generate a cache key. This ensures that vectors from different backends (e.g., Azure OpenAI vs. local LiteLLM) remain isolated in the SQLite cache, preventing semantic mixing of incompatible embedding spaces.