# Which LLM Providers Can Be Used with Code-Graph-RAG for Cypher Query Generation?

> Discover which LLM providers work with Code-Graph-RAG for Cypher query generation. Explore integrations with OpenAI, Anthropic, Google Vertex AI, and more.

- Repository: [Vitali Avagyan/code-graph-rag](https://github.com/vitali87/code-graph-rag)
- Tags: how-to-guide
- Published: 2026-08-12

---

**Code-Graph-RAG supports seven LLM providers—including OpenAI, Anthropic, Google Vertex AI, Azure OpenAI, Ollama, MiniMax, and LiteLLM—through a unified provider interface defined in [`codebase_rag/providers/base.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/providers/base.py).**

The open-source **Code-Graph-RAG** repository (vitali87/code-graph-rag) abstracts LLM interactions behind a pluggable provider architecture that enables seamless switching between cloud APIs and local models for **Cypher query generation**. By implementing a common interface, the library allows developers to generate Neo4j graph queries using everything from GPT-4 to locally hosted Llama instances without modifying application logic.

## Supported LLM Providers for Cypher Generation

The provider system centers on the abstract `ModelProvider` class defined in [`codebase_rag/providers/base.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/providers/base.py). Each concrete implementation registers itself automatically via the `@register_provider` decorator, making it available to the `CypherGenerator` class in [`codebase_rag/services/llm.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/services/llm.py).

The following seven providers are bundled with the repository:

- **GoogleProvider** – Integrates with Google Vertex AI models using the official Vertex AI client (defined at [`codebase_rag/providers/base.py#L57`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/providers/base.py#L57))
- **OpenAIProvider** – Connects to OpenAI's chat completion endpoints for GPT-4, GPT-4o, and ChatGPT models (defined at [`codebase_rag/providers/base.py#L128`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/providers/base.py#L128))
- **OllamaProvider** – Interfaces with local or remote Ollama servers hosting open-source models like Llama and Mistral (defined at [`codebase_rag/providers/base.py#L158`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/providers/base.py#L158))
- **AnthropicProvider** – Sends requests to Anthropic's Claude model family (defined at [`codebase_rag/providers/base.py#L190`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/providers/base.py#L190))
- **AzureOpenAIProvider** – Targets Azure's hosted OpenAI Service for enterprise deployments (defined at [`codebase_rag/providers/base.py#L222`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/providers/base.py#L222))
- **MiniMaxProvider** – Calls the MiniMax API for LLM inference (defined at [`codebase_rag/providers/base.py#L262`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/providers/base.py#L262))
- **LiteLLMProvider** – Acts as a thin wrapper around the `litellm` library, proxying requests to any provider supported by LiteLLM including Cohere and Mistral AI (defined at [`codebase_rag/providers/litellm.py#L14`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/providers/litellm.py#L14))

## Configuring Providers in CypherGenerator

To instantiate a generator, pass a provider specification string in the format `provider_name:model_name` to the `CypherGenerator` constructor. The library parses this string to select the appropriate registered provider from `PROVIDER_REGISTRY`.

Required parameters vary by provider:

- Cloud providers (OpenAI, Anthropic, Google) require an `api_key`
- Ollama requires an `endpoint` URL (defaults to `http://localhost:11434`)
- Azure OpenAI requires Azure-specific configuration (endpoint and deployment)

### Example: OpenAI GPT-4o

```python
from codebase_rag.services.llm import CypherGenerator

cypher_gen = CypherGenerator(
    provider="openai:gpt-4o",
    api_key="YOUR_OPENAI_API_KEY",
)

query = cypher_gen.generate(
    "Which Python functions call `process_data`?",
    active_projects=["my_project"]
)
print(query)

```

### Example: Local Ollama Instance

```python
from codebase_rag.services.llm import CypherGenerator

cypher_gen = CypherGenerator(
    provider="ollama:llama3.1:8b",
    endpoint="http://localhost:11434",
)

query = cypher_gen.generate(
    "List all classes that inherit from `BaseHandler`.",
    active_projects=["my_project"]
)
print(query)

```

### Example: LiteLLM Proxy for Anthropic

```python
from codebase_rag.services.llm import CypherGenerator

cypher_gen = CypherGenerator(
    provider="litellm:anthropic:claude-3-5-sonnet-20240620",
    api_key="YOUR_ANTHROPIC_API_KEY",
)

query = cypher_gen.generate(
    "Find modules that import `numpy` and define a function named `compute`.",
    active_projects=["analytics"]
)
print(query)

```

## Provider Registration Architecture

The extensibility of Code-Graph-RAG relies on the `PROVIDER_REGISTRY` dictionary and the `@register_provider` decorator in [`codebase_rag/providers/base.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/providers/base.py). When Python imports the provider modules, the decorator automatically adds each class to the registry using a normalized name key (e.g., `"openai"`, `"ollama"`).

The `CypherGenerator` resolves the provider string by splitting on the colon character and looking up the first segment in `PROVIDER_REGISTRY`. This design allows the CLI entry point in [`codebase_rag/cli.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/cli.py) to accept any registered provider via the `--cypher-model` flag without hardcoding provider-specific logic.

## Summary

- Code-Graph-RAG bundles seven concrete provider implementations: **GoogleProvider**, **OpenAIProvider**, **OllamaProvider**, **AnthropicProvider**, **AzureOpenAIProvider**, **MiniMaxProvider**, and **LiteLLMProvider**
- All providers implement the abstract `ModelProvider` interface defined in [`codebase_rag/providers/base.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/providers/base.py) and register automatically via the `@register_provider` decorator
- Configuration uses the syntax `provider:model` (e.g., `"openai:gpt-4o"`, `"ollama:llama3.1:8b"`) passed to the `CypherGenerator` constructor
- The `LiteLLMProvider` extends compatibility to dozens of additional providers through the LiteLLM proxy library
- Source files: [`codebase_rag/providers/base.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/providers/base.py) (base classes), [`codebase_rag/providers/litellm.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/providers/litellm.py) (LiteLLM wrapper), [`codebase_rag/services/llm.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/services/llm.py) (generator logic)

## Frequently Asked Questions

### How do I add a custom LLM provider to Code-Graph-RAG?

Create a class that inherits from `ModelProvider` in [`codebase_rag/providers/base.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/providers/base.py) and implement the required abstract methods for authentication and completion requests. Apply the `@register_provider` decorator with your provider name key, then reference it as `your_provider:model_name` in the `CypherGenerator` configuration.

### Can I use local models with Code-Graph-RAG without internet access?

Yes. The **OllamaProvider** supports fully local inference by connecting to an Ollama server running on your machine or network. Specify the model tag (e.g., `llama3.1:8b`) and local endpoint URL to generate Cypher queries without sending code context to external APIs.

### What is the difference between OpenAIProvider and AzureOpenAIProvider?

`OpenAIProvider` connects directly to OpenAI's public API endpoints using standard API keys, while `AzureOpenAIProvider` integrates with Microsoft's Azure OpenAI Service, requiring Azure-specific authentication tokens and deployment names. Both implement the same interface but handle authentication headers and base URLs differently according to their respective cloud requirements.

### How does LiteLLMProvider extend compatibility beyond the built-in providers?

`LiteLLMProvider` acts as a universal adapter by wrapping the `litellm` Python library, which normalizes API calls across 100+ providers including Cohere, Mistral AI, and AWS Bedrock. This allows Code-Graph-RAG to access models from providers not explicitly implemented in [`base.py`](https://github.com/vitali87/code-graph-rag/blob/main/base.py) by routing requests through LiteLLM's unified interface.