# Configuring LLM Providers (Anthropic, Google, Ollama) for Code-Graph-RAG Orchestration

> Effortlessly configure LLM providers like Anthropic, Google, and Ollama for Code-Graph-RAG orchestration. Set environment variables and let the config.py validate and instantiate your chosen provider.

- Repository: [Vitali Avagyan/code-graph-rag](https://github.com/vitali87/code-graph-rag)
- Tags: how-to-guide
- Published: 2026-08-15

---

**Set environment variables in `ORCHESTRATOR_*` and `CYPHER_*` namespaces, then let [`codebase_rag/config.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/config.py) validate and instantiate the appropriate provider from `anthropic`, `google`, or `ollama`.**

Code‑Graph‑RAG from `vitali87/code-graph-rag` is a flexible retrieval‑augmented generation system that lets you plug in any LLM following the *pydantic‑ai* contract. The **orchestrator**—the model that generates Cypher queries, prompts, and reasoning logic—is driven by a small, declarative configuration layer. This guide explains how to configure Anthropic, Google (GLA or Vertex), and Ollama providers for both orchestration and Cypher generation.

---

## Provider Selection via Environment Variables

All provider configuration flows through the Pydantic settings class in [`codebase_rag/config.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/config.py)【/codebase_rag/config.py†L166-L173】. The system recognizes these variable namespaces:

| Variable | Purpose | Example Values |
|----------|---------|--------------|
| `ORCHESTRATOR_PROVIDER` | Provider for the main reasoning LLM | `anthropic`, `google`, `ollama` |
| `ORCHESTRATOR_MODEL` | Model identifier | `claude-3-5-sonnet`, `gemini-2.5-pro`, `llama3.2` |
| `ORCHESTRATOR_API_KEY` | API key for cloud providers | `sk-ant-...`, `AIza...` |
| `ORCHESTRATOR_ENDPOINT` | Base URL for local/custom endpoints | `http://localhost:11434/v1` |
| `ORCHESTRATOR_PROVIDER_TYPE` | Google subtype (`gla` or `vertex`) | `gla`, `vertex` |
| `ORCHESTRATOR_PROJECT_ID` / `ORCHESTRATOR_REGION` | Required for Vertex AI | `my-project`, `us-central1` |
| `ORCHESTRATOR_SERVICE_ACCOUNT_FILE` | GCP service account path | [`/path/to/sa.json`](https://github.com/vitali87/code-graph-rag/blob/main//path/to/sa.json) |

A parallel `CYPHER_*` namespace exists for the Cypher‑generation model. The validator enforces required fields per provider—for example, Anthropic demands an API key while Vertex requires a project ID.

---

## Provider Implementations in [`providers/base.py`](https://github.com/vitali87/code-graph-rag/blob/main/providers/base.py)

The concrete provider classes inherit from the abstract `ModelProvider` in [`codebase_rag/providers/base.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/providers/base.py). Each implements `create_model()` to return a pydantic‑ai compatible instance.

### AnthropicProvider

**`AnthropicProvider`** wraps `pydantic_ai.models.anthropic.AnthropicModel` and requires `api_key`. It constructs the model with `AnthropicModelSettings`【/codebase_rag/providers/base.py†L190-L219】.

```python

# Required: ORCHESTRATOR_API_KEY

ORCHESTRATOR_PROVIDER=anthropic
ORCHESTRATOR_MODEL=claude-3-5-sonnet

```

### GoogleProvider

**`GoogleProvider`** supports two sub‑types controlled by `GoogleProviderType` defined in [`codebase_rag/constants/providers.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/constants/providers.py)【/codebase_rag/constants/providers.py†L36-L45】:

- **`gla`** – Google LLM API via `PydanticGoogleProvider`
- **`vertex`** – Vertex AI via `GoogleCloudProvider`

Both paths inject `GoogleModelSettings`【/codebase_rag/providers/base.py†L70-L125】.

```dotenv

# Google LLM API (GLA)

ORCHESTRATOR_PROVIDER=google
ORCHESTRATOR_PROVIDER_TYPE=gla
ORCHESTRATOR_MODEL=gemini-2.5-pro
ORCHESTRATOR_API_KEY=AIza...

# Vertex AI

ORCHESTRATOR_PROVIDER=google
ORCHESTRATOR_PROVIDER_TYPE=vertex
ORCHESTRATOR_MODEL=gemini-2.5-pro
ORCHESTRATOR_PROJECT_ID=my-project
ORCHESTRATOR_REGION=us-central1
ORCHESTRATOR_SERVICE_ACCOUNT_FILE=/path/to/sa.json

```

### OllamaProvider

**`OllamaProvider`** is a lightweight local provider. The endpoint defaults to `settings.ollama_endpoint` (see the property in [`config.py`](https://github.com/vitali87/code-graph-rag/blob/main/config.py)【/codebase_rag/config.py†L191-L197】) but can be overridden. On initialization, it validates reachability via `check_ollama_running`【/codebase_rag/providers/base.py†L168-L178】【/codebase_rag/providers/base.py†L354-L357】.

```dotenv
ORCHESTRATOR_PROVIDER=ollama
ORCHESTRATOR_MODEL=llama3.2
ORCHESTRATOR_ENDPOINT=http://localhost:11434/v1

```

All providers are registered in a mapping so `get_provider()` resolves strings like `"anthropic"` or `"ollama"` to the correct class【/codebase_rag/providers/base.py†L302-L304】.

---

## Orchestrator Wiring in [`main.py`](https://github.com/vitali87/code-graph-rag/blob/main/main.py)

During startup, [`codebase_rag/main.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/main.py) loads the active configuration:

```python
from codebase_rag import settings, cs
orchestrator_cfg = settings.active_orchestrator_config

```

It then materializes the LLM:

```python
model = orchestrator_cfg.provider.create_model(
    orchestrator_cfg.model_id,
    endpoint=orchestrator_cfg.endpoint,
)

```

The resulting model passes to [`codebase_rag/services/llm.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/services/llm.py), which calls `model.generate(...)` per query. Provider initialization failures raise `LLMGenerationError` with the original exception preserved【/codebase_rag/services/llm.py†L182-L190】.

---

## Mixing Providers for Cost Optimization

Code‑Graph‑RAG allows **different providers for orchestrator and Cypher**. A typical hybrid setup pairs a capable cloud model for reasoning with a local Ollama instance for cheap Cypher generation:

```dotenv
ORCHESTRATOR_PROVIDER=google
ORCHESTRATOR_MODEL=gemini-2.5-pro
ORCHESTRATOR_API_KEY=YOUR_GOOGLE_KEY
ORCHESTRATOR_PROVIDER_TYPE=gla

CYPHER_PROVIDER=ollama
CYPHER_MODEL=llama3.2
CYPHER_ENDPOINT=http://localhost:11434/v1

```

The test suite [`tests/test_provider_configuration.py`](https://github.com/vitali87/code-graph-rag/blob/main/tests/test_provider_configuration.py) validates these mixed configurations【/codebase_rag/tests/test_provider_configuration.py†L17-L38】.

---

## Programmatic Configuration

Bypass environment variables by constructing `ModelConfig` directly and invoking `_update_single_model_setting`:

```python
from codebase_rag.config import ModelConfig, cs
from codebase_rag.main import _update_single_model_setting

# Anthropic orchestrator

_update_single_model_setting(
    cs.ModelRole.ORCHESTRATOR,
    ModelConfig(
        provider="anthropic",
        model_id="claude-3-5-sonnet",
        api_key="sk-ant-test-key"
    )
)

# Ollama for Cypher

_update_single_model_setting(
    cs.ModelRole.CYPHER,
    ModelConfig(
        provider="ollama",
        model_id="llama3.2",
        endpoint="http://localhost:11434/v1"
    )
)

```

---

## Complete Configuration Examples

### Minimal Anthropic Setup

```dotenv
ORCHESTRATOR_PROVIDER=anthropic
ORCHESTRATOR_MODEL=claude-3-5-sonnet
ORCHESTRATOR_API_KEY=sk-ant-test-key

```

```bash
python -m codebase_rag.main query "Find all functions that call process_data"

```

### Google Vertex AI

```dotenv
ORCHESTRATOR_PROVIDER=google
ORCHESTRATOR_MODEL=gemini-2.5-pro
ORCHESTRATOR_PROVIDER_TYPE=vertex
ORCHESTRATOR_PROJECT_ID=my-gcp-project
ORCHESTRATOR_REGION=us-central1
ORCHESTRATOR_SERVICE_ACCOUNT_FILE=/path/to/sa.json

```

```python
from codebase_rag.main import run_query

result = run_query("Show the call graph for DataLoader.load")
print(result)

```

### Ollama-Only Cypher Generation

```dotenv
CYPHER_PROVIDER=ollama
CYPHER_MODEL=llama3.2
CYPHER_ENDPOINT=http://localhost:11434/v1

```

```python
from codebase_rag.main import run_query

result = run_query("List all tables that store user profiles", cypher_role="cypher")
print(result.cypher)  # generated locally

```

---

## Summary

- **Configuration hub**: [`codebase_rag/config.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/config.py) defines `ORCHESTRATOR_*` and `CYPHER_*` variables with Pydantic validation【/codebase_rag/config.py†L166-L173】
- **Provider implementations**: `AnthropicProvider`, `GoogleProvider`, and `OllamaProvider` live in [`codebase_rag/providers/base.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/providers/base.py)【/codebase_rag/providers/base.py†L70-L219】
- **Google sub‑types**: `GoogleProviderType` enum selects GLA versus Vertex【/codebase_rag/constants/providers.py†L36-L45】
- **Mixed deployments**: Cloud orchestrator + local Cypher provider validated in [`tests/test_provider_configuration.py`](https://github.com/vitali87/code-graph-rag/blob/main/tests/test_provider_configuration.py)【/codebase_rag/tests/test_provider_configuration.py†L17-L38】
- **Programmatic control**: `ModelConfig` + `_update_single_model_setting` bypasses environment variables entirely

---

## Frequently Asked Questions

### What happens if I omit `ORCHESTRATOR_API_KEY` for Anthropic?

The Pydantic validator in [`codebase_rag/config.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/config.py) raises a validation error at startup because `AnthropicProvider.create_model()` requires the key to instantiate `AnthropicModel`【/codebase_rag/providers/base.py†L190-L219】.

### Can I use the same Ollama endpoint for both orchestrator and Cypher?

Yes. Set both `ORCHESTRATOR_PROVIDER=ollama` and `CYPHER_PROVIDER=ollama` with identical or separate `*_ENDPOINT` values. The provider performs reachability checks via `check_ollama_running`【/codebase_rag/providers/base.py†L354-L357】.

### How do I switch between Google GLA and Vertex?

Set `ORCHESTRATOR_PROVIDER_TYPE=gla` for the standard Google LLM API (API key only) or `vertex` for Vertex AI (requires project ID, region, and optionally a service account file)【/codebase_rag/constants/providers.py†L36-L45】.

### Is there a default fallback provider?

No explicit default is hard-coded. If environment variables are absent, the Pydantic settings class raises validation errors. For testing, the internal test suite uses explicit `ModelConfig` construction to avoid environment dependencies【/codebase_rag/tests/test_provider_configuration.py†L17-L38】.