# How to Configure code-graph-rag with Ollama for Local Models

> Configure code-graph-rag with Ollama for local models by setting ORCHESTRATOR_PROVIDER to ollama and ORCHESTRATOR_MODEL to your desired model. Run LLM requests locally without an API key.

- Repository: [Vitali Avagyan/code-graph-rag](https://github.com/vitali87/code-graph-rag)
- Tags: how-to-guide
- Published: 2026-09-05

---

**Set `ORCHESTRATOR_PROVIDER=ollama` and `ORCHESTRATOR_MODEL=<model>` in a `.env` file to route all LLM requests to a local Ollama instance running on `http://localhost:11434`, requiring no external API key.**

code-graph-rag is an open-source retrieval-augmented generation framework for codebases that supports multiple LLM providers. When you configure code-graph-rag with Ollama for local models, you enable fully offline, privacy-preserving code analysis without relying on cloud-based APIs.

## Configuration Architecture Overview

The framework treats Ollama as a first-class local provider. Configuration resolution happens through Pydantic's `AppConfig` in [`codebase_rag/config.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/config.py), which loads environment variables from a `.env` file at the project root (lines 20-26).

The **OLLAMA_BASE_URL** setting defaults to `http://localhost:11434` (line 92). The configuration class exposes an `ollama_endpoint` property that automatically appends the `/v1` suffix to create an OpenAI-compatible API URL (lines 94-96).

The provider implementation resides in [`codebase_rag/providers/base.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/providers/base.py). The **OllamaProvider** class constructs the model client using the resolved endpoint and uses the fixed API key string `"ollama"` for authentication (lines 97-119).

## Step-by-Step Setup Guide

### Install and Start Ollama

First, ensure the Ollama daemon is installed and running on your machine.

```bash
curl -fsSL https://ollama.com/install.sh | sh
ollama serve

```

Pull the model you intend to use:

```bash
ollama pull llama3

```

### Create the Environment Configuration

Create a `.env` file in the project root directory with the following settings:

```text
ORCHESTRATOR_PROVIDER=ollama
ORCHESTRATOR_MODEL=llama3

```

The `ORCHESTRATOR_API_KEY` variable is optional. When the provider is set to `ollama`, the framework automatically supplies the placeholder key `"ollama"` internally, eliminating the need for external API credentials.

### Verify and Run

With the configuration in place, the `cgr` CLI automatically routes requests to your local Ollama instance:

```bash

# Index a repository

cgr index /path/to/repo

# Query the codebase

cgr ask "Explain the authentication flow in src/auth.py"

```

## Customizing the Ollama Endpoint

If Ollama runs on a different host, behind a proxy, or on a non-standard port, override the default endpoint by setting **OLLAMA_BASE_URL**:

```text
OLLAMA_BASE_URL=http://192.168.1.10:11434
ORCHESTRATOR_PROVIDER=ollama
ORCHESTRATOR_MODEL=codellama

```

The `ollama_endpoint` property in [`codebase_rag/config.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/config.py) dynamically constructs the final URL by appending `/v1` to your custom base URL, ensuring compatibility with OpenAI-compatible client libraries.

## Programmatic Configuration Example

For library usage, the configuration is accessible through the settings singleton:

```python
from codebase_rag.config import settings

# Verify provider configuration

assert settings.ORCHESTRATOR_PROVIDER == "ollama"
assert settings.ORCHESTRATOR_MODEL == "llama3"

# Access the resolved endpoint (e.g., http://localhost:11434/v1)

print(settings.ollama_endpoint)

# The orchestrator initializes OllamaProvider automatically

# using the resolved settings

```

## Summary

- Configuration resides in **[`codebase_rag/config.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/config.py)**, utilizing Pydantic's `AppConfig` to load `.env` variables.
- Set **`ORCHESTRATOR_PROVIDER=ollama`** to activate the local provider; **`ORCHESTRATOR_MODEL`** specifies which local model to query.
- The default endpoint is **`http://localhost:11434`** with **`/v1`** appended automatically via the `ollama_endpoint` property.
- **No API key is required**; the `OllamaProvider` uses the fixed string `"ollama"` for authentication.
- Override **`OLLAMA_BASE_URL`** to connect to remote Ollama instances or custom ports.

## Frequently Asked Questions

### Do I need an API key to use Ollama with code-graph-rag?

No. According to the implementation in [`codebase_rag/providers/base.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/providers/base.py), the `OllamaProvider` class automatically uses the fixed API key string `"ollama"` (lines 97-119). You can omit the `ORCHESTRATOR_API_KEY` variable entirely from your environment configuration.

### How do I change the port if localhost:11434 is unavailable?

Set the `OLLAMA_BASE_URL` environment variable in your `.env` file to point to your custom host and port, such as `http://localhost:11435`. The framework automatically handles the `/v1` suffix through the `ollama_endpoint` property defined in [`codebase_rag/config.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/config.py) (lines 94-96).

### Which Ollama models are compatible with code-graph-rag?

Any model installed in your local Ollama registry is compatible. Simply set `ORCHESTRATOR_MODEL` to the exact model name as listed by the `ollama list` command (e.g., `llama3`, `codellama`, `mistral`, or `deepseek-coder`).

### Why am I getting connection errors when running cgr commands?

Ensure the Ollama daemon is actively running (`ollama serve`) and accessible at the URL specified in `OLLAMA_BASE_URL` (defaulting to `http://localhost:11434`). The `OllamaProvider` validates server connectivity during client initialization in [`base.py`](https://github.com/vitali87/code-graph-rag/blob/main/base.py), and failures typically indicate the server is not reachable at the configured endpoint.