# How to Configure an LLM Provider in Cognee: Environment Setup Guide

> Easily configure an LLM provider in Cognee using environment variables. Switch between OpenAI, Anthropic, or Ollama effortlessly without code changes. Get started with our setup guide.

- Repository: [Topoteretes/cognee](https://github.com/topoteretes/cognee)
- Tags: getting-started
- Published: 2026-03-16

---

**Cognee centralizes LLM configuration through environment variables read by the `LLMConfig` class, allowing you to switch between providers like OpenAI, Anthropic, or Ollama by setting `LLM_PROVIDER`, `LLM_MODEL`, and `LLM_API_KEY` without changing application code.**

Cognee (topoteretes/cognee) unifies LLM configuration in a single settings layer that propagates to every graph generation, entity extraction, and RAG component. Understanding how to configure an LLM provider in Cognee ensures your pipelines use the correct model for embedding and inference tasks across the entire framework.


## How Cognee's LLM Configuration Works

### The Configuration Layer (LLMConfig)

The configuration entry point is `cognee.infrastructure.llm.config.LLMConfig` defined in [`cognee/infrastructure/llm/config.py`](https://github.com/topoteretes/cognee/blob/main/cognee/infrastructure/llm/config.py). This class inherits from Pydantic's `BaseSettings` and reads environment variables at startup to validate and expose LLM settings.

The following variables control provider selection:

- **LLM_PROVIDER** – The provider name (`openai`, `anthropic`, `gemini`, `ollama`, `mistral`, `bedrock`, etc.)
- **LLM_MODEL** – The specific model identifier (e.g., `gpt-4o-mini`, `claude-3-5-sonnet`)
- **LLM_API_KEY** – The authentication secret required by the provider
- **LLM_ENDPOINT** – Optional URL for self-hosted services (Ollama, local vLLM)

If required variables are missing, `LLMConfig` raises a clear validation error indicating exactly which environment variable is absent.


### The Provider Factory

Downstream components do not read environment variables directly. Instead, they import `LLMProvider` from [`cognee/tasks/translation/providers/llm_provider.py`](https://github.com/topoteretes/cognee/blob/main/cognee/tasks/translation/providers/llm_provider.py). This factory class instantiates a concrete client using the **Litellm-Instructor** adapter based on the values stored in `LLMConfig`.

Because all LLM-driven tasks (entity extraction, graph generation, search) consume this centralized provider, switching from OpenAI to a local Ollama instance requires only changing environment variables—no code modifications needed.


### Observability Integration

Every LLM call emits OpenTelemetry trace attributes defined in [`cognee/infrastructure/llm/structured_output_framework/litellm_instructor/llm/generic_llm_api/adapter.py`](https://github.com/topoteretes/cognee/blob/main/cognee/infrastructure/llm/structured_output_framework/litellm_instructor/llm/generic_llm_api/adapter.py). Specifically, the framework adds:

- `cognee.llm.provider` – The active provider name
- `cognee.llm.model` – The specific model identifier

This gives you trace-level visibility into which provider and model handled each request across distributed pipelines.


## Setting Up Environment Variables

### Using a .env File

The recommended approach is creating a `.env` file in your project root (copy from the repository's `/.env.example`). Cognee automatically loads these values via `BaseSettings`.

```dotenv

# .env configuration

LLM_PROVIDER=openai
LLM_MODEL=gpt-4o-mini
LLM_API_KEY=sk-XXXXXXXXXXXXXXXXXXXXXXXXXXXX

# Required only for self-hosted providers like Ollama

LLM_ENDPOINT=http://localhost:11434/v1

```

### Provider-Specific Configuration Examples

**OpenAI**
- `LLM_PROVIDER=openai`
- `LLM_MODEL=gpt-4o-mini`
- `LLM_API_KEY=sk-...`

**Anthropic**
- `LLM_PROVIDER=anthropic`
- `LLM_MODEL=claude-3-5-sonnet`
- `LLM_API_KEY=sk-ant-...`

**Ollama (Local)**
- `LLM_PROVIDER=ollama`
- `LLM_MODEL=llama3.1:8b`
- `LLM_ENDPOINT=http://localhost:11434/v1`
- `LLM_API_KEY` can be set to any non-empty placeholder if required


## Programmatic and CLI Configuration

### Runtime Overrides in Python

You can override configuration programmatically before initializing components. This is useful for testing or dynamic provider selection.

```python
import os
from cognee.infrastructure.llm.config import LLMConfig

# Set environment variables before instantiating config

os.environ["LLM_PROVIDER"] = "ollama"
os.environ["LLM_MODEL"] = "llama3.1:8b"
os.environ["LLM_ENDPOINT"] = "http://localhost:11434/v1"

# Validate and load configuration

config = LLMConfig()
print(f"Provider: {config.LLM_PROVIDER}, Model: {config.LLM_MODEL}")

```

### One-Off CLI Execution

For single command execution without permanent configuration files, prefix the command with environment variables:

```bash
LLM_PROVIDER=anthropic LLM_MODEL=claude-3-5-sonnet LLM_API_KEY=your_key \
cognee-cli add "Your data chunk to process"

```


## Accessing the Configured LLM in Your Code

Once environment variables are set, retrieve the client through the provider factory or use the high-level gateway for structured output:

```python
from cognee.tasks.translation.providers.llm_provider import LLMProvider
from cognee.infrastructure.llm.LLMGateway import LLMGateway
from pydantic import BaseModel

class SummaryOutput(BaseModel):
    summary: str
    key_points: list[str]

# Factory provides the configured client

provider = LLMProvider()
client = provider.get_client()

# Generate structured output using the configured provider

response = await LLMGateway.acreate_structured_output(
    user_prompt="Summarize the following text",
    system_prompt="You are a concise technical summarizer.",
    output_schema=SummaryOutput,
    text="Long document content here..."
)
print(response)

```

The `LLMGateway` class in [`cognee/infrastructure/llm/LLMGateway.py`](https://github.com/topoteretes/cognee/blob/main/cognee/infrastructure/llm/LLMGateway.py) provides async methods like `acreate_structured_output` and `arun` that automatically respect the `LLMConfig` settings.


## API and MCP Server Integration

The same environment variable configuration propagates to Cognee's public interfaces:

- **REST API endpoints** (`/v1/add`, `/v1/cognify`, `/v1/search`) documented in [`cognee/api/v1/add/add.py`](https://github.com/topoteretes/cognee/blob/main/cognee/api/v1/add/add.py) and [`cognee/api/v1/search/search.py`](https://github.com/topoteretes/cognee/blob/main/cognee/api/v1/search/search.py) expose `LLM_PROVIDER` settings in their OpenAPI schemas
- **MCP Server** ([`cognee-mcp/src/server.py`](https://github.com/topoteretes/cognee/blob/main/cognee-mcp/src/server.py)) forwards these LLM settings to the Model Context Protocol server, ensuring consistent provider usage across tool integrations


## Summary

- **Centralized config**: `LLMConfig` in [`cognee/infrastructure/llm/config.py`](https://github.com/topoteretes/cognee/blob/main/cognee/infrastructure/llm/config.py) reads `LLM_PROVIDER`, `LLM_MODEL`, `LLM_API_KEY`, and optional `LLM_ENDPOINT` from environment variables
- **Factory pattern**: `LLMProvider` in [`cognee/tasks/translation/providers/llm_provider.py`](https://github.com/topoteretes/cognee/blob/main/cognee/tasks/translation/providers/llm_provider.py) creates the concrete Litellm client used by all tasks
- **Observability**: Traces include `cognee.llm.provider` and `cognee.llm.model` attributes for debugging
- **Zero-code switching**: Change providers by updating environment variables; no code changes required in pipelines or API routes


## Frequently Asked Questions

### What environment variables are required to configure an LLM provider in Cognee?

You must set `LLM_PROVIDER` (e.g., `openai`, `anthropic`), `LLM_MODEL` (e.g., `gpt-4o-mini`), and `LLM_API_KEY`. For self-hosted providers like Ollama, you must also specify `LLM_ENDPOINT` with the local API URL.

### Can I switch LLM providers without restarting my Cognee application?

`LLMConfig` reads environment variables at instantiation time using Pydantic `BaseSettings`. To switch providers in a running process, you must create a new `LLMConfig` instance after updating `os.environ`, or restart the application to pick up new `.env` values.

### Does Cognee support local or self-hosted LLMs?

Yes. Set `LLM_PROVIDER=ollama` (or another local-compatible provider) and configure `LLM_ENDPOINT` to point to your local inference server (e.g., `http://localhost:11434/v1` for Ollama). The Litellm adapter handles the translation between Cognee's structured output requirements and local API formats.

### How can I verify which LLM provider is currently active in my traces?

Check the OpenTelemetry trace attributes `cognee.llm.provider` and `cognee.llm.model` emitted by the adapter in [`cognee/infrastructure/llm/structured_output_framework/litellm_instructor/llm/generic_llm_api/adapter.py`](https://github.com/topoteretes/cognee/blob/main/cognee/infrastructure/llm/structured_output_framework/litellm_instructor/llm/generic_llm_api/adapter.py). Alternatively, print an `LLMConfig` instance to inspect the loaded `LLM_PROVIDER` and `LLM_MODEL` values at runtime.