# Hindsight LLM Providers: Complete Configuration Guide for OpenAI, Groq, Gemini, and 100+ Models

> Explore Hindsight's complete LLM provider configuration guide. Seamlessly set up OpenAI, Groq, Gemini, Ollama, Vertex AI, and 100+ models via LiteLLM with environment variables or profile files.

- Repository: [vectorize-io/hindsight](https://github.com/vectorize-io/hindsight)
- Tags: how-to-guide
- Published: 2026-03-13

---

**Hindsight supports five built-in providers (OpenAI, Groq, Gemini, Ollama, Vertex AI) via the embedded CLI, and over 100 additional providers through its LiteLLM integration, all configured via environment variables or profile files.**

Hindsight by vectorize-io is an open-source memory layer for LLM applications that integrates with multiple model providers through two distinct mechanisms. Understanding which LLM providers are supported by Hindsight and how to configure them is essential for deploying the memory retention system across different AI infrastructure stacks.

## Built-In LLM Providers in Hindsight

The embedded CLI (`hindsight-embed`) ships with first-class support for five curated providers defined in [`hindsight-embed/hindsight_embed/cli.py`](https://github.com/vectorize-io/hindsight/blob/main/hindsight-embed/hindsight_embed/cli.py). These providers are hardcoded in the `PROVIDER_DEFAULTS` dictionary at lines 46-53:

```python
PROVIDER_DEFAULTS = {
    "openai": ("openai", get_default_model_for_provider("openai"), "OPENAI_API_KEY"),
    "groq": ("groq", get_default_model_for_provider("groq"), "GROQ_API_KEY"),
    "gemini": ("gemini", get_default_model_for_provider("gemini"), "GEMINI_API_KEY"),
    "ollama": ("ollama", get_default_model_for_provider("ollama"), None),
    "vertexai": ("vertexai", get_default_model_for_provider("vertexai"), None),
}

```

Each entry contains a tuple with the provider identifier, default model resolution function, and the required environment variable name for API authentication.

### Provider Defaults and Fallback Models

When `get_default_model_for_provider()` is called without a specific configuration, it references `hindsight_api.config.PROVIDER_DEFAULT_MODELS`. If the provider lacks a specific entry in that configuration, Hindsight falls back to `"gpt-4o-mini"` as the default model.

The `get_config()` function (lines 33-41 of [`cli.py`](https://github.com/vectorize-io/hindsight/blob/main/cli.py)) assembles the final configuration dictionary used by the daemon and forwarder:

```python
{
    "llm_api_key": <key-or-None>,
    "llm_provider": provider,
    "llm_model": model,
    "bank_id": <bank>,
}

```

### Environment Variable Requirements

Built-in providers require specific environment variables based on the third tuple item in `PROVIDER_DEFAULTS`:

- **OpenAI**: Set `OPENAI_API_KEY`
- **Groq**: Set `GROQ_API_KEY` 
- **Gemini**: Set `GEMINI_API_KEY`
- **Ollama**: No API key required (runs locally)
- **Vertex AI**: No explicit key variable; uses GCP Application Default Credentials via `GOOGLE_APPLICATION_CREDENTIALS`

Set the active provider using `HINDSIGHT_API_LLM_PROVIDER` (defaults to `openai` if unspecified).

## LiteLLM Integration for Extended Provider Support

Beyond the five built-in providers, Hindsight supports **100+ LLM providers** through the LiteLLM integration located in `hindsight-integrations/litellm/`. The module docstring in [`hindsight_litellm/__init__.py`](https://github.com/vectorize-io/hindsight/blob/main/hindsight_litellm/__init__.py) explicitly states this capability:

> "This package provides automatic memory integration for any LLM provider supported by LiteLLM (100+ providers including OpenAI, Anthropic, Groq, Azure, AWS Bedrock, Google Vertex AI, and more)."

### How the LiteLLM Callback Works

The integration lives in [`hindsight_litellm/callbacks.py`](https://github.com/vectorize-io/hindsight/blob/main/hindsight_litellm/callbacks.py), which implements `HindsightCallback()`. When initialized, this callback reads the same environment variables used by the embedded CLI (`HINDSIGHT_API_LLM_PROVIDER`, `HINDSIGHT_API_LLM_API_KEY`). Because LiteLLM internally maps provider strings to their respective SDKs, you can pass any LiteLLM-compatible identifier (e.g., `"anthropic"`, `"azure"`, `"bedrock"`) without modifying Hindsight source code.

```python
import os
import litellm
from hindsight_litellm import callbacks

os.environ["HINDSIGHT_API_LLM_PROVIDER"] = "anthropic"
os.environ["HINDSIGHT_API_LLM_API_KEY"] = "my-anthropic-key"

litellm.success_callback = [callbacks.HindsightCallback()]

resp = litellm.completion(
    model="claude-3-5-sonnet-20240620",
    messages=[{"role": "user", "content": "What is the weather?"}]
)

```

## Configuration Mechanisms

Hindsight provides four primary methods for configuring LLM providers, implemented across [`cli.py`](https://github.com/vectorize-io/hindsight/blob/main/cli.py) and [`profile_manager.py`](https://github.com/vectorize-io/hindsight/blob/main/profile_manager.py):

### Environment Variables and Profiles

Direct environment variables offer immediate configuration for single sessions:
- `HINDSIGHT_API_LLM_PROVIDER` — selects the provider (e.g., `groq`, `gemini`)
- `HINDSIGHT_API_LLM_API_KEY` — passes the authentication key
- `HINDSIGHT_API_LLM_MODEL` — overrides the default model selection

For persistent configuration, Hindsight uses the profile system in [`hindsight-embed/hindsight_embed/profile_manager.py`](https://github.com/vectorize-io/hindsight/blob/main/hindsight-embed/hindsight_embed/profile_manager.py). Profile files store environment variables in:
- Default profile: `~/.hindsight/embed`
- Named profiles: `~/.hindsight/profiles/<name>.env`

### Interactive CLI Configuration

The [`cli.py`](https://github.com/vectorize-io/hindsight/blob/main/cli.py) module provides `_do_configure_interactive()` and `do_profile_command()` functions for guided setup. These commands prompt for provider selection, API key entry, model specification, and bank ID assignment, then persist the values to the appropriate profile file.

```bash

# Create a profile with explicit provider configuration

hindsight-embed profile create my-app --port 9100 \
    --env HINDSIGHT_API_LLM_PROVIDER=openai \
    --env HINDSIGHT_API_LLM_API_KEY=sk-...

# Verify configuration

hindsight-embed profile show -o json

# Execute with profile

hindsight-embed -p my-app memory retain default "User prefers dark mode"

```

### Programmatic Configuration with LiteLLM

When using the LiteLLM integration, provider arguments passed directly to `litellm.completion()` override environment variables for that specific call. However, Hindsight callbacks still read the env vars when initializing the memory layer.

## Provider-Specific Setup Requirements

Different LLM provider families require distinct authentication approaches within the Hindsight framework:

| Provider | Configuration Method | API Key Variable |
|----------|---------------------|------------------|
| **OpenAI** | Environment or profile | `OPENAI_API_KEY` |
| **Groq** | Environment or profile | `GROQ_API_KEY` |
| **Gemini** | Environment or profile | `GEMINI_API_KEY` |
| **Ollama** | Local endpoint only | None required |
| **Vertex AI** | GCP ADC | `GOOGLE_APPLICATION_CREDENTIALS` (implicit) |
| **Other LiteLLM providers** (Anthropic, Azure, Bedrock) | Environment passthrough | Provider-specific (e.g., `ANTHROPIC_API_KEY`) |

For LiteLLM-specific providers not in the built-in list, set `HINDSIGHT_API_LLM_PROVIDER` to the LiteLLM identifier and export the provider's standard API key variable. Hindsight forwards the key from `HINDSIGHT_API_LLM_API_KEY` when specified, or you can set the provider-native variable name directly in your profile.

## Summary

- **Five built-in providers** (`openai`, `groq`, `gemini`, `ollama`, `vertexai`) are defined in [`hindsight-embed/hindsight_embed/cli.py`](https://github.com/vectorize-io/hindsight/blob/main/hindsight-embed/hindsight_embed/cli.py) via `PROVIDER_DEFAULTS`
- **100+ additional providers** are accessible through the LiteLLM integration in `hindsight-integrations/litellm/`
- **Configuration uses environment variables** (`HINDSIGHT_API_LLM_PROVIDER`, `HINDSIGHT_API_LLM_API_KEY`) or persistent profile files managed by [`profile_manager.py`](https://github.com/vectorize-io/hindsight/blob/main/profile_manager.py)
- **Fallback behavior** defaults to `openai` provider and `gpt-4o-mini` model when unspecified
- **Local deployment** via Ollama requires no API keys, while Vertex AI relies on GCP service account credentials

## Frequently Asked Questions

### How do I configure Ollama or Vertex AI without API keys?

Set `HINDSIGHT_API_LLM_PROVIDER=ollama` or `vertexai` respectively. For Ollama, no authentication is required as it runs on your local machine. For Vertex AI, ensure your environment has valid Google Cloud Platform Application Default Credentials configured via `GOOGLE_APPLICATION_CREDENTIALS` rather than a static API key.

### Can I use Anthropic Claude or Azure OpenAI with Hindsight?

Yes. Install the LiteLLM integration and set `HINDSIGHT_API_LLM_PROVIDER` to `"anthropic"` or `"azure"`. Export the provider's standard API key variable (e.g., `ANTHROPIC_API_KEY` or `AZURE_API_KEY`) or pass it via `HINDSIGHT_API_LLM_API_KEY`. The callback implementation in [`hindsight_litellm/callbacks.py`](https://github.com/vectorize-io/hindsight/blob/main/hindsight_litellm/callbacks.py) handles the provider-specific routing automatically.

### Where does Hindsight store profile configurations?

Profile configurations are stored in the `~/.hindsight/` directory according to [`hindsight-embed/hindsight_embed/profile_manager.py`](https://github.com/vectorize-io/hindsight/blob/main/hindsight-embed/hindsight_embed/profile_manager.py). The default profile uses `~/.hindsight/embed`, while named profiles are saved as `~/.hindsight/profiles/<name>.env`. These files contain key-value pairs for environment variables like `HINDSIGHT_API_LLM_PROVIDER` and `HINDSIGHT_API_LLM_API_KEY`.

### What happens if I don't specify an LLM provider?

If `HINDSIGHT_API_LLM_PROVIDER` is unset, the `get_config()` function in [`cli.py`](https://github.com/vectorize-io/hindsight/blob/main/cli.py) defaults to `"openai"`. Similarly, if `get_default_model_for_provider()` cannot resolve a specific model for the selected provider, it falls back to `"gpt-4o-mini"`. This ensures Hindsight remains functional with minimal configuration while allowing explicit overrides.