Which LLM Providers Can Be Used with Code-Graph-RAG for Cypher Query Generation?

Code-Graph-RAG supports seven LLM providers—including OpenAI, Anthropic, Google Vertex AI, Azure OpenAI, Ollama, MiniMax, and LiteLLM—through a unified provider interface defined in codebase_rag/providers/base.py.

The open-source Code-Graph-RAG repository (vitali87/code-graph-rag) abstracts LLM interactions behind a pluggable provider architecture that enables seamless switching between cloud APIs and local models for Cypher query generation. By implementing a common interface, the library allows developers to generate Neo4j graph queries using everything from GPT-4 to locally hosted Llama instances without modifying application logic.

Supported LLM Providers for Cypher Generation

The provider system centers on the abstract ModelProvider class defined in codebase_rag/providers/base.py. Each concrete implementation registers itself automatically via the @register_provider decorator, making it available to the CypherGenerator class in codebase_rag/services/llm.py.

The following seven providers are bundled with the repository:

• GoogleProvider – Integrates with Google Vertex AI models using the official Vertex AI client (defined at codebase_rag/providers/base.py#L57)
• OpenAIProvider – Connects to OpenAI's chat completion endpoints for GPT-4, GPT-4o, and ChatGPT models (defined at codebase_rag/providers/base.py#L128)
• OllamaProvider – Interfaces with local or remote Ollama servers hosting open-source models like Llama and Mistral (defined at codebase_rag/providers/base.py#L158)
• AnthropicProvider – Sends requests to Anthropic's Claude model family (defined at codebase_rag/providers/base.py#L190)
• AzureOpenAIProvider – Targets Azure's hosted OpenAI Service for enterprise deployments (defined at codebase_rag/providers/base.py#L222)
• MiniMaxProvider – Calls the MiniMax API for LLM inference (defined at codebase_rag/providers/base.py#L262)
• LiteLLMProvider – Acts as a thin wrapper around the litellm library, proxying requests to any provider supported by LiteLLM including Cohere and Mistral AI (defined at codebase_rag/providers/litellm.py#L14)

Configuring Providers in CypherGenerator

To instantiate a generator, pass a provider specification string in the format provider_name:model_name to the CypherGenerator constructor. The library parses this string to select the appropriate registered provider from PROVIDER_REGISTRY.

Required parameters vary by provider:

• Cloud providers (OpenAI, Anthropic, Google) require an api_key
• Ollama requires an endpoint URL (defaults to http://localhost:11434)
• Azure OpenAI requires Azure-specific configuration (endpoint and deployment)

Example: OpenAI GPT-4o

from codebase_rag.services.llm import CypherGenerator

cypher_gen = CypherGenerator(
    provider="openai:gpt-4o",
    api_key="YOUR_OPENAI_API_KEY",
)

query = cypher_gen.generate(
    "Which Python functions call `process_data`?",
    active_projects=["my_project"]
)
print(query)

Example: Local Ollama Instance

from codebase_rag.services.llm import CypherGenerator

cypher_gen = CypherGenerator(
    provider="ollama:llama3.1:8b",
    endpoint="http://localhost:11434",
)

query = cypher_gen.generate(
    "List all classes that inherit from `BaseHandler`.",
    active_projects=["my_project"]
)
print(query)

Example: LiteLLM Proxy for Anthropic

from codebase_rag.services.llm import CypherGenerator

cypher_gen = CypherGenerator(
    provider="litellm:anthropic:claude-3-5-sonnet-20240620",
    api_key="YOUR_ANTHROPIC_API_KEY",
)

query = cypher_gen.generate(
    "Find modules that import `numpy` and define a function named `compute`.",
    active_projects=["analytics"]
)
print(query)

Provider Registration Architecture

The extensibility of Code-Graph-RAG relies on the PROVIDER_REGISTRY dictionary and the @register_provider decorator in codebase_rag/providers/base.py. When Python imports the provider modules, the decorator automatically adds each class to the registry using a normalized name key (e.g., "openai", "ollama").

The CypherGenerator resolves the provider string by splitting on the colon character and looking up the first segment in PROVIDER_REGISTRY. This design allows the CLI entry point in codebase_rag/cli.py to accept any registered provider via the --cypher-model flag without hardcoding provider-specific logic.

Summary

• Code-Graph-RAG bundles seven concrete provider implementations: GoogleProvider, OpenAIProvider, OllamaProvider, AnthropicProvider, AzureOpenAIProvider, MiniMaxProvider, and LiteLLMProvider
• All providers implement the abstract ModelProvider interface defined in codebase_rag/providers/base.py and register automatically via the @register_provider decorator
• Configuration uses the syntax provider:model (e.g., "openai:gpt-4o", "ollama:llama3.1:8b") passed to the CypherGenerator constructor
• The LiteLLMProvider extends compatibility to dozens of additional providers through the LiteLLM proxy library
• Source files: codebase_rag/providers/base.py (base classes), codebase_rag/providers/litellm.py (LiteLLM wrapper), codebase_rag/services/llm.py (generator logic)

Frequently Asked Questions

How do I add a custom LLM provider to Code-Graph-RAG?

Create a class that inherits from ModelProvider in codebase_rag/providers/base.py and implement the required abstract methods for authentication and completion requests. Apply the @register_provider decorator with your provider name key, then reference it as your_provider:model_name in the CypherGenerator configuration.

Can I use local models with Code-Graph-RAG without internet access?

Yes. The OllamaProvider supports fully local inference by connecting to an Ollama server running on your machine or network. Specify the model tag (e.g., llama3.1:8b) and local endpoint URL to generate Cypher queries without sending code context to external APIs.

What is the difference between OpenAIProvider and AzureOpenAIProvider?

OpenAIProvider connects directly to OpenAI's public API endpoints using standard API keys, while AzureOpenAIProvider integrates with Microsoft's Azure OpenAI Service, requiring Azure-specific authentication tokens and deployment names. Both implement the same interface but handle authentication headers and base URLs differently according to their respective cloud requirements.

How does LiteLLMProvider extend compatibility beyond the built-in providers?

LiteLLMProvider acts as a universal adapter by wrapping the litellm Python library, which normalizes API calls across 100+ providers including Cohere, Mistral AI, and AWS Bedrock. This allows Code-Graph-RAG to access models from providers not explicitly implemented in base.py by routing requests through LiteLLM's unified interface.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}