Hindsight LLM Providers: Complete Configuration Guide for OpenAI, Groq, Gemini, and 100+ Models

Hindsight supports five built-in providers (OpenAI, Groq, Gemini, Ollama, Vertex AI) via the embedded CLI, and over 100 additional providers through its LiteLLM integration, all configured via environment variables or profile files.

Hindsight by vectorize-io is an open-source memory layer for LLM applications that integrates with multiple model providers through two distinct mechanisms. Understanding which LLM providers are supported by Hindsight and how to configure them is essential for deploying the memory retention system across different AI infrastructure stacks.

Built-In LLM Providers in Hindsight

The embedded CLI (hindsight-embed) ships with first-class support for five curated providers defined in hindsight-embed/hindsight_embed/cli.py. These providers are hardcoded in the PROVIDER_DEFAULTS dictionary at lines 46-53:

PROVIDER_DEFAULTS = {
    "openai": ("openai", get_default_model_for_provider("openai"), "OPENAI_API_KEY"),
    "groq": ("groq", get_default_model_for_provider("groq"), "GROQ_API_KEY"),
    "gemini": ("gemini", get_default_model_for_provider("gemini"), "GEMINI_API_KEY"),
    "ollama": ("ollama", get_default_model_for_provider("ollama"), None),
    "vertexai": ("vertexai", get_default_model_for_provider("vertexai"), None),
}

Each entry contains a tuple with the provider identifier, default model resolution function, and the required environment variable name for API authentication.

Provider Defaults and Fallback Models

When get_default_model_for_provider() is called without a specific configuration, it references hindsight_api.config.PROVIDER_DEFAULT_MODELS. If the provider lacks a specific entry in that configuration, Hindsight falls back to "gpt-4o-mini" as the default model.

The get_config() function (lines 33-41 of cli.py) assembles the final configuration dictionary used by the daemon and forwarder:

{
    "llm_api_key": <key-or-None>,
    "llm_provider": provider,
    "llm_model": model,
    "bank_id": <bank>,
}

Environment Variable Requirements

Built-in providers require specific environment variables based on the third tuple item in PROVIDER_DEFAULTS:

• OpenAI: Set OPENAI_API_KEY
• Groq: Set GROQ_API_KEY
• Gemini: Set GEMINI_API_KEY
• Ollama: No API key required (runs locally)
• Vertex AI: No explicit key variable; uses GCP Application Default Credentials via GOOGLE_APPLICATION_CREDENTIALS

Set the active provider using HINDSIGHT_API_LLM_PROVIDER (defaults to openai if unspecified).

LiteLLM Integration for Extended Provider Support

Beyond the five built-in providers, Hindsight supports 100+ LLM providers through the LiteLLM integration located in hindsight-integrations/litellm/. The module docstring in hindsight_litellm/__init__.py explicitly states this capability:

"This package provides automatic memory integration for any LLM provider supported by LiteLLM (100+ providers including OpenAI, Anthropic, Groq, Azure, AWS Bedrock, Google Vertex AI, and more)."

How the LiteLLM Callback Works

The integration lives in hindsight_litellm/callbacks.py, which implements HindsightCallback(). When initialized, this callback reads the same environment variables used by the embedded CLI (HINDSIGHT_API_LLM_PROVIDER, HINDSIGHT_API_LLM_API_KEY). Because LiteLLM internally maps provider strings to their respective SDKs, you can pass any LiteLLM-compatible identifier (e.g., "anthropic", "azure", "bedrock") without modifying Hindsight source code.

import os
import litellm
from hindsight_litellm import callbacks

os.environ["HINDSIGHT_API_LLM_PROVIDER"] = "anthropic"
os.environ["HINDSIGHT_API_LLM_API_KEY"] = "my-anthropic-key"

litellm.success_callback = [callbacks.HindsightCallback()]

resp = litellm.completion(
    model="claude-3-5-sonnet-20240620",
    messages=[{"role": "user", "content": "What is the weather?"}]
)

Configuration Mechanisms

Hindsight provides four primary methods for configuring LLM providers, implemented across cli.py and profile_manager.py:

Environment Variables and Profiles

Direct environment variables offer immediate configuration for single sessions:

• HINDSIGHT_API_LLM_PROVIDER — selects the provider (e.g., groq, gemini)
• HINDSIGHT_API_LLM_API_KEY — passes the authentication key
• HINDSIGHT_API_LLM_MODEL — overrides the default model selection

For persistent configuration, Hindsight uses the profile system in hindsight-embed/hindsight_embed/profile_manager.py. Profile files store environment variables in:

• Default profile: ~/.hindsight/embed
• Named profiles: ~/.hindsight/profiles/<name>.env

Interactive CLI Configuration

The cli.py module provides _do_configure_interactive() and do_profile_command() functions for guided setup. These commands prompt for provider selection, API key entry, model specification, and bank ID assignment, then persist the values to the appropriate profile file.

# Create a profile with explicit provider configuration

hindsight-embed profile create my-app --port 9100 \
    --env HINDSIGHT_API_LLM_PROVIDER=openai \
    --env HINDSIGHT_API_LLM_API_KEY=sk-...

# Verify configuration

hindsight-embed profile show -o json

# Execute with profile

hindsight-embed -p my-app memory retain default "User prefers dark mode"

Programmatic Configuration with LiteLLM

When using the LiteLLM integration, provider arguments passed directly to litellm.completion() override environment variables for that specific call. However, Hindsight callbacks still read the env vars when initializing the memory layer.

Provider-Specific Setup Requirements

Different LLM provider families require distinct authentication approaches within the Hindsight framework:

Provider	Configuration Method	API Key Variable
OpenAI	Environment or profile	OPENAI_API_KEY
Groq	Environment or profile	GROQ_API_KEY
Gemini	Environment or profile	GEMINI_API_KEY
Ollama	Local endpoint only	None required
Vertex AI	GCP ADC	GOOGLE_APPLICATION_CREDENTIALS (implicit)
Other LiteLLM providers (Anthropic, Azure, Bedrock)	Environment passthrough	Provider-specific (e.g., ANTHROPIC_API_KEY)

For LiteLLM-specific providers not in the built-in list, set HINDSIGHT_API_LLM_PROVIDER to the LiteLLM identifier and export the provider's standard API key variable. Hindsight forwards the key from HINDSIGHT_API_LLM_API_KEY when specified, or you can set the provider-native variable name directly in your profile.

Summary

• Five built-in providers (openai, groq, gemini, ollama, vertexai) are defined in hindsight-embed/hindsight_embed/cli.py via PROVIDER_DEFAULTS
• 100+ additional providers are accessible through the LiteLLM integration in hindsight-integrations/litellm/
• Configuration uses environment variables (HINDSIGHT_API_LLM_PROVIDER, HINDSIGHT_API_LLM_API_KEY) or persistent profile files managed by profile_manager.py
• Fallback behavior defaults to openai provider and gpt-4o-mini model when unspecified
• Local deployment via Ollama requires no API keys, while Vertex AI relies on GCP service account credentials

Frequently Asked Questions

How do I configure Ollama or Vertex AI without API keys?

Set HINDSIGHT_API_LLM_PROVIDER=ollama or vertexai respectively. For Ollama, no authentication is required as it runs on your local machine. For Vertex AI, ensure your environment has valid Google Cloud Platform Application Default Credentials configured via GOOGLE_APPLICATION_CREDENTIALS rather than a static API key.

Can I use Anthropic Claude or Azure OpenAI with Hindsight?

Yes. Install the LiteLLM integration and set HINDSIGHT_API_LLM_PROVIDER to "anthropic" or "azure". Export the provider's standard API key variable (e.g., ANTHROPIC_API_KEY or AZURE_API_KEY) or pass it via HINDSIGHT_API_LLM_API_KEY. The callback implementation in hindsight_litellm/callbacks.py handles the provider-specific routing automatically.

Where does Hindsight store profile configurations?

Profile configurations are stored in the ~/.hindsight/ directory according to hindsight-embed/hindsight_embed/profile_manager.py. The default profile uses ~/.hindsight/embed, while named profiles are saved as ~/.hindsight/profiles/<name>.env. These files contain key-value pairs for environment variables like HINDSIGHT_API_LLM_PROVIDER and HINDSIGHT_API_LLM_API_KEY.

What happens if I don't specify an LLM provider?

If HINDSIGHT_API_LLM_PROVIDER is unset, the get_config() function in cli.py defaults to "openai". Similarly, if get_default_model_for_provider() cannot resolve a specific model for the selected provider, it falls back to "gpt-4o-mini". This ensures Hindsight remains functional with minimal configuration while allowing explicit overrides.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}