How to Set a Custom LLM Endpoint for Cognee: 3 Configuration Methods

Set a custom LLM endpoint in Cognee by using the environment variable LLM_ENDPOINT, calling cognee.config.set_llm_endpoint(), or running cognee config set llm_endpoint <url> — all three methods update the centralized LLMConfig object consumed by every LLM adapter.

Cognee supports routing language model requests to any OpenAI-compatible server, including Ollama, self-hosted inference services, or private cloud endpoints. Configuring a custom LLM endpoint allows you to use alternative providers without modifying the core topoteretes/cognee source code. This guide explains the three configuration interfaces and how they interact with the LLM infrastructure layer.

How Cognee Manages LLM Endpoints

Cognee uses a centralized configuration pattern based on Pydantic Settings. The LLMConfig class in cognee/infrastructure/llm/config.py inherits from pydantic_settings.BaseSettings and automatically reads environment variables at import time.

The configuration object exposes the llm_endpoint field (default empty string), which adapters consume when initializing their respective SDK clients. When you change the endpoint via any supported method, you mutate this singleton configuration instance, immediately affecting all subsequent LLM calls through the LLMGateway.

Method 1: Environment Variable

The simplest approach for containerized or scripted deployments is setting the LLM_ENDPOINT environment variable. At process startup, LLMConfig reads this value via BaseSettings and populates the llm_endpoint attribute.

Add the variable to your shell or .env file:

export LLM_ENDPOINT="http://localhost:11434/v1"

Or in a .env file:

LLM_ENDPOINT="http://localhost:11434/v1"

When Cognee imports, cognee/infrastructure/llm/config.py (lines 44-46) captures this value automatically.

Method 2: Python API

For dynamic configuration within scripts or notebooks, use the static method set_llm_endpoint() exposed through the cognee.config module. This method retrieves the singleton LLMConfig instance via get_llm_config() and updates the llm_endpoint attribute at runtime.

import cognee
from cognee.infrastructure.llm.config import get_llm_config

# Set a custom endpoint (e.g., local Ollama or private server)

cognee.config.set_llm_endpoint("http://my-llm.local:8000/v1")

# Verify the configuration

print(get_llm_config().llm_endpoint)  # → http://my-llm.local:8000/v1

The implementation resides in cognee/api/v1/config/config.py (lines 84-92), where the static method handles the mutation of the global config object.

Method 3: CLI

The Cognee CLI provides a config set command that maps keys to the corresponding Python API methods. To change the LLM endpoint from the terminal:

# Set the endpoint

cognee config set llm_endpoint http://localhost:11434/v1

# Verify the current value

cognee config get llm_endpoint

In cognee/cli/commands/config_command.py, the config_key_mappings dictionary links the string "llm_endpoint" to the static setter method, ensuring CLI changes propagate to the same underlying configuration object used by the Python API.

How Adapters Consume the Custom Endpoint

Every LLM adapter in Cognee fetches the current configuration using get_llm_config() and passes llm_config.llm_endpoint to the provider's SDK constructor. For example, in cognee/infrastructure/llm/structured_output_framework/litellm_instructor/llm/openai/adapter.py (lines 158-161), the adapter initializes the client as:

OpenAI(base_url=self.endpoint, ...)

Similarly, the Ollama adapter in cognee/infrastructure/llm/structured_output_framework/litellm_instructor/llm/ollama/adapter.py follows the same pattern, reading the endpoint from the shared config. This architecture ensures that once you set a custom LLM endpoint, all providers (OpenAI, Ollama, Mistral, etc.) route requests to your specified URL without additional per-adapter configuration.

Summary

• Environment Variable: Set LLM_ENDPOINT before startup; LLMConfig reads it automatically via Pydantic Settings in cognee/infrastructure/llm/config.py.
• Python API: Call cognee.config.set_llm_endpoint(url) to update the runtime singleton configuration.
• CLI: Use cognee config set llm_endpoint <url> for command-line workflows; the mapping is defined in cognee/cli/commands/config_command.py.
• Adapter Integration: All LLM clients retrieve the endpoint from get_llm_config().llm_endpoint when building their respective SDK instances.

Frequently Asked Questions

What file stores the LLM endpoint configuration in Cognee?

The LLMConfig class in cognee/infrastructure/llm/config.py stores the endpoint as a Pydantic Settings field. This class inherits from BaseSettings, enabling automatic environment variable loading and providing the singleton instance accessed via get_llm_config().

Can I switch LLM endpoints after Cognee has already initialized?

Yes. Calling cognee.config.set_llm_endpoint() mutates the global LLMConfig singleton at runtime. Any LLM adapter created after this call will use the new endpoint value, though existing client instances may retain the previous configuration depending on their initialization timing.

Does setting a custom endpoint work with all LLM providers in Cognee?

Yes. The endpoint configuration is provider-agnostic. Both the OpenAI adapter (openai/adapter.py) and Ollama adapter (ollama/adapter.py) consume llm_config.llm_endpoint when constructing their clients, allowing you to point any supported provider at a custom base URL.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}