How to Configure code-graph-rag with Ollama for Local Models

Set ORCHESTRATOR_PROVIDER=ollama and ORCHESTRATOR_MODEL=<model> in a .env file to route all LLM requests to a local Ollama instance running on http://localhost:11434, requiring no external API key.

code-graph-rag is an open-source retrieval-augmented generation framework for codebases that supports multiple LLM providers. When you configure code-graph-rag with Ollama for local models, you enable fully offline, privacy-preserving code analysis without relying on cloud-based APIs.

Configuration Architecture Overview

The framework treats Ollama as a first-class local provider. Configuration resolution happens through Pydantic's AppConfig in codebase_rag/config.py, which loads environment variables from a .env file at the project root (lines 20-26).

The OLLAMA_BASE_URL setting defaults to http://localhost:11434 (line 92). The configuration class exposes an ollama_endpoint property that automatically appends the /v1 suffix to create an OpenAI-compatible API URL (lines 94-96).

The provider implementation resides in codebase_rag/providers/base.py. The OllamaProvider class constructs the model client using the resolved endpoint and uses the fixed API key string "ollama" for authentication (lines 97-119).

Step-by-Step Setup Guide

Install and Start Ollama

First, ensure the Ollama daemon is installed and running on your machine.

curl -fsSL https://ollama.com/install.sh | sh
ollama serve

Pull the model you intend to use:

ollama pull llama3

Create the Environment Configuration

Create a .env file in the project root directory with the following settings:

ORCHESTRATOR_PROVIDER=ollama
ORCHESTRATOR_MODEL=llama3

The ORCHESTRATOR_API_KEY variable is optional. When the provider is set to ollama, the framework automatically supplies the placeholder key "ollama" internally, eliminating the need for external API credentials.

Verify and Run

With the configuration in place, the cgr CLI automatically routes requests to your local Ollama instance:

# Index a repository

cgr index /path/to/repo

# Query the codebase

cgr ask "Explain the authentication flow in src/auth.py"

Customizing the Ollama Endpoint

If Ollama runs on a different host, behind a proxy, or on a non-standard port, override the default endpoint by setting OLLAMA_BASE_URL:

OLLAMA_BASE_URL=http://192.168.1.10:11434
ORCHESTRATOR_PROVIDER=ollama
ORCHESTRATOR_MODEL=codellama

The ollama_endpoint property in codebase_rag/config.py dynamically constructs the final URL by appending /v1 to your custom base URL, ensuring compatibility with OpenAI-compatible client libraries.

Programmatic Configuration Example

For library usage, the configuration is accessible through the settings singleton:

from codebase_rag.config import settings

# Verify provider configuration

assert settings.ORCHESTRATOR_PROVIDER == "ollama"
assert settings.ORCHESTRATOR_MODEL == "llama3"

# Access the resolved endpoint (e.g., http://localhost:11434/v1)

print(settings.ollama_endpoint)

# The orchestrator initializes OllamaProvider automatically

# using the resolved settings

Summary

• Configuration resides in codebase_rag/config.py, utilizing Pydantic's AppConfig to load .env variables.
• Set ORCHESTRATOR_PROVIDER=ollama to activate the local provider; ORCHESTRATOR_MODEL specifies which local model to query.
• The default endpoint is http://localhost:11434 with /v1 appended automatically via the ollama_endpoint property.
• No API key is required; the OllamaProvider uses the fixed string "ollama" for authentication.
• Override OLLAMA_BASE_URL to connect to remote Ollama instances or custom ports.

Frequently Asked Questions

Do I need an API key to use Ollama with code-graph-rag?

No. According to the implementation in codebase_rag/providers/base.py, the OllamaProvider class automatically uses the fixed API key string "ollama" (lines 97-119). You can omit the ORCHESTRATOR_API_KEY variable entirely from your environment configuration.

How do I change the port if localhost:11434 is unavailable?

Set the OLLAMA_BASE_URL environment variable in your .env file to point to your custom host and port, such as http://localhost:11435. The framework automatically handles the /v1 suffix through the ollama_endpoint property defined in codebase_rag/config.py (lines 94-96).

Which Ollama models are compatible with code-graph-rag?

Any model installed in your local Ollama registry is compatible. Simply set ORCHESTRATOR_MODEL to the exact model name as listed by the ollama list command (e.g., llama3, codellama, mistral, or deepseek-coder).

Why am I getting connection errors when running cgr commands?

Ensure the Ollama daemon is actively running (ollama serve) and accessible at the URL specified in OLLAMA_BASE_URL (defaulting to http://localhost:11434). The OllamaProvider validates server connectivity during client initialization in base.py, and failures typically indicate the server is not reachable at the configured endpoint.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}