Which LLMs Are Compatible with code‑graph‑rag? Complete Provider Guide

code‑graph‑rag supports seven LLM providers—OpenAI, Anthropic, Google, Azure, Ollama, Minimax, and Litellm‑Proxy—configured via ORCHESTRATOR_PROVIDER and CYPHER_PROVIDER environment variables.

The code‑graph‑rag framework by vitali87 offers a pluggable LLM architecture that lets you swap models without changing application code. Whether you need cloud‑hosted frontier models or local open‑weight alternatives, the provider system abstracts away implementation differences behind a unified interface. This guide explains every supported provider, how to configure them, and where to find the relevant source code.

Supported LLM Providers for code‑graph‑rag

The complete list of compatible providers is defined in codebase_rag/constants/providers.py and documented in docs/getting-started/configuration.md. Each provider supports distinct model families for two separate roles:

• Orchestrator (ORCHESTRATOR_PROVIDER): Generates natural‑language responses to user queries
• Cypher Generator (CYPHER_PROVIDER): Translates natural language into Neo4j Cypher graph queries

Provider	Typical Models	Best For
OpenAI	gpt‑4, gpt‑4‑turbo, gpt‑5.6‑terra, gpt‑5.6‑luna	High‑quality reasoning and Cypher generation
Anthropic	claude‑sonnet‑5, claude‑3‑opus‑20240229	Long‑context understanding and safety
Google	gemini‑3.5‑flash, gemini‑1.5‑pro, gemini‑2.5‑pro	Multimodal capabilities and cost efficiency
Azure	Azure‑hosted OpenAI models (e.g., gpt‑35‑turbo)	Enterprise compliance and private endpoints
Ollama	llama2, mistral, phi‑3, codellama	Fully offline, air‑gapped deployments
Minimax	ab‑cocoa‑v2‑latest, ab‑chat‑v2‑latest	Regional API access
Litellm‑Proxy	Any model via Litellm (Together.ai, DeepSeek, etc.)	Unified gateway to 100+ models

These values are enumerated at line 84–98 of docs/getting-started/configuration.md, where the environment variable schema defines allowed provider strings.

How Provider Selection Works Internally

The configuration layer in codebase_rag/config.py implements a provider factory pattern. On startup, it reads four environment variables:

• ORCHESTRATOR_PROVIDER
• ORCHESTRATOR_MODEL
• CYPHER_PROVIDER
• CYPHER_MODEL

Each provider string maps to a Provider enum member, which instantiates a concrete client object with standardized methods for:

• generate(prompt: str) -> str — text completion
• embed(texts: list[str]) -> list[list[float]] — vector embeddings

Provider‑Specific SDK Implementations

Provider	SDK	Key Methods
OpenAI	openai Python package	openai.ChatCompletion.create, openai.Embedding.create
Anthropic	anthropic	client.completion
Google	google-generativeai	Gemini Pro API
Azure	openai with Azure AD	Endpoint‑prefixed OpenAI calls
Ollama	HTTP REST client	POST /v1/chat/completions to local server
Minimax	Custom HTTP client	Minimax REST API with bearer token
Litellm‑Proxy	OpenAI‑compatible client	Forwards to Litellm gateway endpoint

This abstraction means switching from GPT‑4 to a local Llama model requires only environment variable changes—no code modifications.

Configuration Examples for code‑graph‑rag

OpenAI GPT‑5.6‑terra for Production Orchestration

import os
from codebase_rag.settings import Settings

settings = Settings()
settings.set_orchestrator(
    provider="openai",
    model="gpt-5.6-terra",
    api_key=os.getenv("OPENAI_API_KEY")
)

Local Ollama for Offline Development

settings.set_orchestrator(
    provider="ollama",
    model="llama2:13b",
    api_key=None  # Ollama requires no authentication

)

Different Providers for Separate Roles

# Use Claude for responses, OpenAI for Cypher (optimal cost/quality tradeoff)

settings.set_orchestrator(
    provider="anthropic",
    model="claude-sonnet-5",
    api_key=os.getenv("ANTHROPIC_API_KEY")
)

settings.set_cypher_generator(
    provider="openai",
    model="gpt-4-turbo",
    api_key=os.getenv("OPENAI_API_KEY")
)

These patterns are demonstrated in docs/sdk/overview.md at lines 50–52.

Embedding Provider Configuration

Semantic search in code‑graph‑rag uses a separate embedding provider controlled by CGR_EMBEDDING_PROVIDER and CGR_EMBEDDING_MODEL. All seven providers above support embedding endpoints, though Ollama and OpenAI are most commonly used:

# Environment-based configuration

export CGR_EMBEDDING_PROVIDER=openai
export CGR_EMBEDDING_MODEL=text-embedding-3-large

See docs/sdk/semantic-search.md for complete embedding setup instructions.

Key Source Files for LLM Compatibility

File	Purpose
codebase_rag/config.py	Parses environment variables and constructs provider objects
codebase_rag/constants/providers.py	Defines the Provider enum with all supported values
docs/getting-started/configuration.md	Documents provider options and required environment variables
docs/sdk/overview.md	Programmatic orchestrator configuration examples
docs/sdk/semantic-search.md	Embedding provider setup guide

Summary

• code‑graph‑rag supports seven LLM providers: OpenAI, Anthropic, Google, Azure, Ollama, Minimax, and Litellm‑Proxy
• Dual provider system: ORCHESTRATOR_PROVIDER for responses, CYPHER_PROVIDER for graph query generation
• Zero‑code provider switching: Change models by updating environment variables in codebase_rag/config.py
• Local deployment ready: Ollama integration enables fully offline operation
• Enterprise support: Azure and Litellm‑Proxy provide private endpoints and unified model gateways

Frequently Asked Questions

Can I use different LLMs for orchestration and Cypher generation?

Yes. The ORCHESTRATOR_PROVIDER and CYPHER_PROVIDER variables are independent. Many deployments use a capable model like GPT‑4 or Claude for Cypher generation (where precision matters) and a faster, cheaper model for response generation.

Does code‑graph‑rag work without internet access?

Yes, via Ollama. Configure ORCHESTRATOR_PROVIDER=ollama and CYPHER_PROVIDER=ollama with locally running models. Both Llama‑based and code‑specialized models like CodeLlama work for Cypher generation, though accuracy may vary compared to frontier models.

What is the Litellm‑Proxy provider used for?

Litellm‑Proxy acts as a universal gateway. It allows code‑graph‑rag to access any model supported by the Litellm ecosystem—including DeepSeek, Together.ai, and custom fine‑tunes—through a single OpenAI‑compatible endpoint. This is ideal for organizations standardizing on Litellm for cost tracking and rate limiting.

How do I migrate from OpenAI to Anthropic?

Update four environment variables. Change ORCHESTRATOR_PROVIDER (or CYPHER_PROVIDER) to "anthropic", set ORCHESTRATOR_MODEL to your chosen Claude version, and provide your ANTHROPIC_API_KEY. No code changes are required—the Provider enum in codebase_rag/constants/providers.py handles the client instantiation automatically.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}