# Which LLMs Are Compatible with code‑graph‑rag? Complete Provider Guide

> Discover which LLMs are compatible with code-graph-rag. Explore providers like OpenAI, Anthropic, Google, and more in this comprehensive guide.

- Repository: [Vitali Avagyan/code-graph-rag](https://github.com/vitali87/code-graph-rag)
- Tags: getting-started
- Published: 2026-08-14

---

**code‑graph‑rag supports seven LLM providers—OpenAI, Anthropic, Google, Azure, Ollama, Minimax, and Litellm‑Proxy—configured via `ORCHESTRATOR_PROVIDER` and `CYPHER_PROVIDER` environment variables.**

The **code‑graph‑rag** framework by vitali87 offers a **pluggable LLM architecture** that lets you swap models without changing application code. Whether you need cloud‑hosted frontier models or local open‑weight alternatives, the provider system abstracts away implementation differences behind a unified interface. This guide explains every supported provider, how to configure them, and where to find the relevant source code.

## Supported LLM Providers for code‑graph‑rag

The complete list of compatible providers is defined in [`codebase_rag/constants/providers.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/constants/providers.py) and documented in [`docs/getting-started/configuration.md`](https://github.com/vitali87/code-graph-rag/blob/main/docs/getting-started/configuration.md). Each provider supports distinct model families for two separate roles:

- **Orchestrator** (`ORCHESTRATOR_PROVIDER`): Generates natural‑language responses to user queries
- **Cypher Generator** (`CYPHER_PROVIDER`): Translates natural language into Neo4j Cypher graph queries

| Provider | Typical Models | Best For |
|----------|--------------|----------|
| **OpenAI** | `gpt‑4`, `gpt‑4‑turbo`, `gpt‑5.6‑terra`, `gpt‑5.6‑luna` | High‑quality reasoning and Cypher generation |
| **Anthropic** | `claude‑sonnet‑5`, `claude‑3‑opus‑20240229` | Long‑context understanding and safety |
| **Google** | `gemini‑3.5‑flash`, `gemini‑1.5‑pro`, `gemini‑2.5‑pro` | Multimodal capabilities and cost efficiency |
| **Azure** | Azure‑hosted OpenAI models (e.g., `gpt‑35‑turbo`) | Enterprise compliance and private endpoints |
| **Ollama** | `llama2`, `mistral`, `phi‑3`, `codellama` | Fully offline, air‑gapped deployments |
| **Minimax** | `ab‑cocoa‑v2‑latest`, `ab‑chat‑v2‑latest` | Regional API access |
| **Litellm‑Proxy** | Any model via Litellm (Together.ai, DeepSeek, etc.) | Unified gateway to 100+ models |

These values are enumerated at **line 84–98 of [`docs/getting-started/configuration.md`](https://github.com/vitali87/code-graph-rag/blob/main/docs/getting-started/configuration.md)**, where the environment variable schema defines allowed provider strings.

## How Provider Selection Works Internally

The configuration layer in [`codebase_rag/config.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/config.py) implements a **provider factory pattern**. On startup, it reads four environment variables:

- `ORCHESTRATOR_PROVIDER`
- `ORCHESTRATOR_MODEL`
- `CYPHER_PROVIDER`
- `CYPHER_MODEL`

Each provider string maps to a `Provider` enum member, which instantiates a concrete client object with standardized methods for:

- `generate(prompt: str) -> str` — text completion
- `embed(texts: list[str]) -> list[list[float]]` — vector embeddings

### Provider‑Specific SDK Implementations

| Provider | SDK | Key Methods |
|----------|-----|-------------|
| **OpenAI** | `openai` Python package | `openai.ChatCompletion.create`, `openai.Embedding.create` |
| **Anthropic** | `anthropic` | `client.completion` |
| **Google** | `google-generativeai` | Gemini Pro API |
| **Azure** | `openai` with Azure AD | Endpoint‑prefixed OpenAI calls |
| **Ollama** | HTTP REST client | `POST /v1/chat/completions` to local server |
| **Minimax** | Custom HTTP client | Minimax REST API with bearer token |
| **Litellm‑Proxy** | OpenAI‑compatible client | Forwards to Litellm gateway endpoint |

This abstraction means **switching from GPT‑4 to a local Llama model requires only environment variable changes**—no code modifications.

## Configuration Examples for code‑graph‑rag

### OpenAI GPT‑5.6‑terra for Production Orchestration

```python
import os
from codebase_rag.settings import Settings

settings = Settings()
settings.set_orchestrator(
    provider="openai",
    model="gpt-5.6-terra",
    api_key=os.getenv("OPENAI_API_KEY")
)

```

### Local Ollama for Offline Development

```python
settings.set_orchestrator(
    provider="ollama",
    model="llama2:13b",
    api_key=None  # Ollama requires no authentication

)

```

### Different Providers for Separate Roles

```python

# Use Claude for responses, OpenAI for Cypher (optimal cost/quality tradeoff)

settings.set_orchestrator(
    provider="anthropic",
    model="claude-sonnet-5",
    api_key=os.getenv("ANTHROPIC_API_KEY")
)

settings.set_cypher_generator(
    provider="openai",
    model="gpt-4-turbo",
    api_key=os.getenv("OPENAI_API_KEY")
)

```

These patterns are demonstrated in **[`docs/sdk/overview.md`](https://github.com/vitali87/code-graph-rag/blob/main/docs/sdk/overview.md)** at **lines 50–52**.

## Embedding Provider Configuration

Semantic search in **code‑graph‑rag** uses a separate embedding provider controlled by `CGR_EMBEDDING_PROVIDER` and `CGR_EMBEDDING_MODEL`. All seven providers above support embedding endpoints, though Ollama and OpenAI are most commonly used:

```bash

# Environment-based configuration

export CGR_EMBEDDING_PROVIDER=openai
export CGR_EMBEDDING_MODEL=text-embedding-3-large

```

See **[`docs/sdk/semantic-search.md`](https://github.com/vitali87/code-graph-rag/blob/main/docs/sdk/semantic-search.md)** for complete embedding setup instructions.

## Key Source Files for LLM Compatibility

| File | Purpose |
|------|---------|
| [`codebase_rag/config.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/config.py) | Parses environment variables and constructs provider objects |
| [`codebase_rag/constants/providers.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/constants/providers.py) | Defines the `Provider` enum with all supported values |
| [`docs/getting-started/configuration.md`](https://github.com/vitali87/code-graph-rag/blob/main/docs/getting-started/configuration.md) | Documents provider options and required environment variables |
| [`docs/sdk/overview.md`](https://github.com/vitali87/code-graph-rag/blob/main/docs/sdk/overview.md) | Programmatic orchestrator configuration examples |
| [`docs/sdk/semantic-search.md`](https://github.com/vitali87/code-graph-rag/blob/main/docs/sdk/semantic-search.md) | Embedding provider setup guide |

## Summary

- **code‑graph‑rag** supports **seven LLM providers**: OpenAI, Anthropic, Google, Azure, Ollama, Minimax, and Litellm‑Proxy
- **Dual provider system**: `ORCHESTRATOR_PROVIDER` for responses, `CYPHER_PROVIDER` for graph query generation
- **Zero‑code provider switching**: Change models by updating environment variables in [`codebase_rag/config.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/config.py)
- **Local deployment ready**: Ollama integration enables fully offline operation
- **Enterprise support**: Azure and Litellm‑Proxy provide private endpoints and unified model gateways

## Frequently Asked Questions

### Can I use different LLMs for orchestration and Cypher generation?

**Yes.** The `ORCHESTRATOR_PROVIDER` and `CYPHER_PROVIDER` variables are independent. Many deployments use a capable model like GPT‑4 or Claude for Cypher generation (where precision matters) and a faster, cheaper model for response generation.

### Does code‑graph‑rag work without internet access?

**Yes, via Ollama.** Configure `ORCHESTRATOR_PROVIDER=ollama` and `CYPHER_PROVIDER=ollama` with locally running models. Both Llama‑based and code‑specialized models like CodeLlama work for Cypher generation, though accuracy may vary compared to frontier models.

### What is the Litellm‑Proxy provider used for?

**Litellm‑Proxy acts as a universal gateway.** It allows **code‑graph‑rag** to access any model supported by the Litellm ecosystem—including DeepSeek, Together.ai, and custom fine‑tunes—through a single OpenAI‑compatible endpoint. This is ideal for organizations standardizing on Litellm for cost tracking and rate limiting.

### How do I migrate from OpenAI to Anthropic?

**Update four environment variables.** Change `ORCHESTRATOR_PROVIDER` (or `CYPHER_PROVIDER`) to `"anthropic"`, set `ORCHESTRATOR_MODEL` to your chosen Claude version, and provide your `ANTHROPIC_API_KEY`. No code changes are required—the `Provider` enum in [`codebase_rag/constants/providers.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/constants/providers.py) handles the client instantiation automatically.