# What LLM Models Can I Use with Cognee? A Complete Provider Guide

> Explore Cognee's integration with nine LLM providers like OpenAI, Gemini, Mistral, and AWS Bedrock. Configure your preferred LLM effortlessly in our guide.

- Repository: [Topoteretes/cognee](https://github.com/topoteretes/cognee)
- Tags: getting-started
- Published: 2026-03-16

---

**Cognee supports nine LLM providers including OpenAI, Anthropic, Google Gemini, Mistral, AWS Bedrock, Ollama, Llama-CPP, and custom HTTP endpoints, configurable via environment variables in `LLMConfig`.**

Cognee abstracts Large Language Model access behind a unified provider-model pair architecture. According to the topoteretes/cognee source code, the framework routes all LLM calls through `LLMGateway` while allowing you to swap between cloud APIs and local inference engines without changing application code.

## Supported LLM Providers in Cognee

The `LLMProvider` enum in [`cognee/infrastructure/llm/structured_output_framework/litellm_instructor/llm/get_llm_client.py`](https://github.com/topoteretes/cognee/blob/main/cognee/infrastructure/llm/structured_output_framework/litellm_instructor/llm/get_llm_client.py) defines nine supported providers:

- **OpenAI** – Cloud models including GPT-5-mini, GPT-4o, and GPT-4
- **Anthropic** – Claude 3 Opus, Sonnet, and Haiku models
- **Ollama** – Local models via Ollama server (Llama 3, Mistral, etc.)
- **Gemini** – Google Gemini 1.5 Pro and Flash
- **Mistral** – Mistral Large and open models
- **Bedrock** – AWS Bedrock including Claude and Llama 2
- **Llama-CPP** – Local GGUF models via llama.cpp
- **Custom** – Any OpenAI-compatible HTTP endpoint
- **BAML** – Optional structured output framework (install `cognee[baml]`)

## How Cognee's LLM Abstraction Works

The architecture centers on three core components that make provider switching transparent.

### LLMConfig

[`cognee/infrastructure/llm/config.py`](https://github.com/topoteretes/cognee/blob/main/cognee/infrastructure/llm/config.py) contains the `LLMConfig` dataclass, which parses environment variables and sets defaults. The default configuration uses `openai` as the provider and `openai/gpt-5-mini` as the model.

### Provider Routing

[`cognee/infrastructure/llm/structured_output_framework/litellm_instructor/llm/get_llm_client.py`](https://github.com/topoteretes/cognee/blob/main/cognee/infrastructure/llm/structured_output_framework/litellm_instructor/llm/get_llm_client.py) implements the routing logic. The `get_llm_client` function maps the `LLMProvider` enum to specific adapter classes, returning a client that implements `acreate_structured_output` and `create_transcript` regardless of the underlying service.

### LLMGateway

[`cognee/infrastructure/llm/LLMGateway.py`](https://github.com/topoteretes/cognee/blob/main/cognee/infrastructure/llm/LLMGateway.py) serves as the public façade. Application code calls `LLMGateway.acreate_structured_output()` without knowing which provider is configured, enabling seamless provider switching at runtime.

## Configuring Your LLM Provider

Cognee reads configuration from environment variables or a `.env` file at startup via `LLMConfig`.

### Environment Variables

```bash
LLM_PROVIDER=openai          # Provider enum value

LLM_MODEL=openai/gpt-4o-mini # Model identifier

LLM_API_KEY=sk-...           # API key for cloud providers

LLM_ENDPOINT=                # Optional custom base URL

```

### Local Inference with Ollama

```bash
LLM_PROVIDER=ollama
LLM_MODEL=ollama/llama3
LLM_API_KEY=dummy            # Ollama may not require authentication

```

### AWS Bedrock

```bash
LLM_PROVIDER=bedrock
LLM_MODEL=bedrock/anthropic.claude-v2

# AWS credentials handled separately via boto3

```

### Custom Endpoints

```bash
LLM_PROVIDER=custom
LLM_MODEL=my-company/awesome-llm
LLM_ENDPOINT=https://api.mycompany.com/v1/completions
LLM_API_KEY=my-secret-token

```

## Code Examples

### Generating Structured Output

```python
import asyncio
from cognee.infrastructure.llm.LLMGateway import LLMGateway
from pydantic import BaseModel, Field

class Answer(BaseModel):
    summary: str = Field(..., description="A concise answer")
    confidence: float = Field(..., description="Confidence score 0-1")

async def ask(question: str) -> Answer:
    system_prompt = "You are a helpful assistant. Answer concisely."
    return await LLMGateway.acreate_structured_output(
        text_input=question,
        system_prompt=system_prompt,
        response_model=Answer,
    )

result = asyncio.run(ask("What is the capital of France?"))
print(result.summary)   # → "Paris"

print(result.confidence)  # → 0.99

```

### Switching to Ollama at Runtime

```python
import os
os.environ["LLM_PROVIDER"] = "ollama"
os.environ["LLM_MODEL"] = "ollama/llama3"
os.environ["LLM_API_KEY"] = "dummy"

from cognee.infrastructure.llm.LLMGateway import LLMGateway

async def translate(text: str) -> str:
    return await LLMGateway.acreate_structured_output(
        text_input=text,
        system_prompt="Translate to German, output only the translation.",
        response_model=str,
    )

```

### Using Custom HTTP Endpoints

```python
import os
os.environ.update({
    "LLM_PROVIDER": "custom",
    "LLM_MODEL": "my-company/awesome-llm",
    "LLM_ENDPOINT": "https://api.mycompany.com/v1/completions",
    "LLM_API_KEY": "my-secret-token",
})

from cognee.infrastructure.llm.LLMGateway import LLMGateway

async def simple_chat(message: str) -> str:
    return await LLMGateway.acreate_structured_output(
        text_input=message,
        system_prompt="You are a terse chatbot.",
        response_model=str,
    )

```

## Summary

- Cognee supports **nine LLM providers**: OpenAI, Anthropic, Ollama, Gemini, Mistral, AWS Bedrock, Llama-CPP, Custom endpoints, and BAML.
- Configuration happens via **environment variables** parsed by `LLMConfig` in [`cognee/infrastructure/llm/config.py`](https://github.com/topoteretes/cognee/blob/main/cognee/infrastructure/llm/config.py).
- The **provider-model pair** abstraction in [`get_llm_client.py`](https://github.com/topoteretes/cognee/blob/main/get_llm_client.py) routes requests to the correct adapter while exposing a unified interface through `LLMGateway`.
- You can switch providers at runtime by setting `LLM_PROVIDER`, `LLM_MODEL`, and `LLM_API_KEY` before importing `LLMGateway`.

## Frequently Asked Questions

### What is the default LLM model in Cognee?

The default configuration uses **OpenAI** as the provider and **GPT-5-mini** as the model. This is defined in [`cognee/infrastructure/llm/config.py`](https://github.com/topoteretes/cognee/blob/main/cognee/infrastructure/llm/config.py) at lines 44-45. You can override this by setting the `LLM_PROVIDER` and `LLM_MODEL` environment variables before starting your application.

### Can I use local models with Cognee?

Yes. Cognee supports **Ollama** and **Llama-CPP** for local inference. For Ollama, set `LLM_PROVIDER=ollama` and `LLM_MODEL=ollama/llama3` (or any model installed on your Ollama server). For Llama-CPP, use `LLM_PROVIDER=llama_cpp` and provide the model path via `LLAMA_CPP_MODEL_PATH`.

### How do I switch LLM providers without changing my code?

Cognee's `LLMGateway` façade makes provider switching configuration-driven. Simply update your environment variables or `.env` file with the new `LLM_PROVIDER`, `LLM_MODEL`, and `LLM_API_KEY` values. The `get_llm_client` function in [`cognee/infrastructure/llm/structured_output_framework/litellm_instructor/llm/get_llm_client.py`](https://github.com/topoteretes/cognee/blob/main/cognee/infrastructure/llm/structured_output_framework/litellm_instructor/llm/get_llm_client.py) handles the routing to the appropriate adapter automatically.

### Does Cognee support AWS Bedrock and other enterprise providers?

Yes. Cognee includes native support for **AWS Bedrock** via the `bedrock` provider setting. You can use models like `bedrock/anthropic.claude-v2` or `bedrock/meta.llama2-13b-chat`. Additionally, the `custom` provider allows integration with any OpenAI-compatible HTTP endpoint, making Cognee compatible with Azure OpenAI, vLLM, and other enterprise deployment patterns.