How to Configure Per-Bank LLM Settings with Hierarchical Overrides in Hindsight

You can override LLM provider, model, and service-tier settings for individual memory banks in Hindsight by sending a BankConfigUpdate payload to the /banks/{bank_id}/config endpoint, which merges your changes with global and tenant-level configurations while filtering out credential fields.

Hindsight implements a three-tier configuration system that allows granular control over LLM behavior at the global, tenant, and individual bank levels. This hierarchical override mechanism enables multi-tenant deployments to maintain default settings while allowing specific memory banks to use different models, providers, or service tiers. Understanding how to configure per-bank LLM settings ensures you can optimize performance and cost for specific workloads without affecting the broader system.

Understanding the Three-Tier Configuration Hierarchy

Hindsight stores configuration in three distinct tiers that are merged sequentially on every request. Only fields marked as hierarchical in hindsight-api-slim/hindsight_api/config.py can be overridden at lower levels.

Global Configuration (Environment Variables)

The top tier consists of environment variables such as HINDSIGHT_API_LLM_PROVIDER and HINDSIGHT_API_LLM_MODEL. These define the base configuration for the entire Hindsight instance. The complete list of hierarchical fields available for override is defined by HindsightConfig.get_configurable_fields() in hindsight-api-slim/hindsight_api/config.py (lines 86-98), which includes LLM provider, model, API keys, service tiers, and per-operation overrides like retain_llm_model and reflect_llm_model.

Tenant-Level Overrides (TenantExtension)

In multi-tenant deployments, an optional TenantExtension can provide middle-tier overrides. These apply to all banks within a specific tenant scope. The resolver normalizes environment-style keys (e.g., HINDSIGHT_API_LLM_PROVIDER) to Python field names (e.g., llm_provider) and filters to hierarchical fields only, as implemented in hindsight-api-slim/hindsight_api/config_resolver.py (lines 65-73).

Bank-Specific Settings (banks.config)

The lowest tier stores overrides in the banks.config JSONB column in the database. These settings apply exclusively to the individual memory bank and take precedence over global and tenant configurations. The system applies the same key normalization and field filtering used at the tenant level (lines 66-70 in config_resolver.py).

How Configuration Resolution Works

When Hindsight processes a request, it resolves the effective configuration through a strict merge sequence:

1. Start with the full global config created from environment variables via HindsightConfig.from_env(), stored as self._global_config.
2. Apply tenant overrides if a TenantExtension is present, keeping only hierarchical fields.
3. Apply bank overrides from the banks.config JSONB column, normalizing keys and filtering to hierarchical fields.
4. Instantiate a fresh HindsightConfig object from the merged dictionary and return it to the caller (lines 86-89 in config_resolver.py).

The ConfigResolver class in hindsight-api-slim/hindsight_api/config_resolver.py handles this logic through methods like resolve_full_config() and get_bank_config(). If you attempt to set a non-hierarchical field or any credential field (API keys, URLs), the resolver raises a ValueError with a clear rejection message (lines 94-101).

Updating Per-Bank LLM Settings via API

To modify LLM settings for a specific bank, send a PATCH request with a BankConfigUpdate payload. The Pydantic model in hindsight-clients/python/hindsight_client_api/models/bank_config_update.py (lines 25-31) validates that only hierarchical fields are present and explicitly rejects credential fields.

Using the Python Client

from hindsight_client_api import HindsightClient
from hindsight_client_api.models import BankConfigUpdate

# Initialize client pointing to your Hindsight server

client = HindsightClient(base_url="https://hindsight.example.com", api_key="YOUR_ADMIN_KEY")

# Build overrides using Python field names or env-style keys

updates = {
    "llm_provider": "anthropic",
    "llm_model": "claude-3-5-sonnet-20240620",
    "llm_openai_service_tier": None,
    "llm_groq_service_tier": "flex",
    "retain_llm_model": "gpt-4o-mini",
    "reflect_llm_model": "gpt-4o",
}

payload = BankConfigUpdate(updates=updates)

# Send the patch request to /banks/{bank_id}/config

response = client.banks.update_bank_config(
    bank_id="user-123",
    bank_config_update=payload,
)

print("Update status:", response.status_code)  # 200 indicates success

Direct HTTP Implementation

For scripts or curl usage, call the endpoint directly:

import json
import requests

url = "https://hindsight.example.com/banks/team-42/config"
headers = {
    "Authorization": "Bearer ADMIN_KEY",
    "Content-Type": "application/json"
}
payload = {
    "updates": {
        "llm_provider": "anthropic",
        "llm_model": "claude-3-5-sonnet-20240620",
        "HINDSIGHT_API_LLM_GROQ_SERVICE_TIER": "flex"  # Env-style keys also accepted

    }
}

r = requests.patch(url, headers=headers, data=json.dumps(payload))
print(r.status_code, r.text)

The FastAPI route handling this endpoint resides in hindsight-api-slim/hindsight_api/api/http.py (lines 1028-1035 and 3982-3988), which invokes ConfigResolver.update_bank_config() to process the changes.

Retrieving Effective Configuration

To debug or inspect the active settings for a bank, retrieve the effective configuration via the GET /banks/{bank_id}/config endpoint. This returns only hierarchical, non-credential fields.

# Using the Python client

config = client.banks.get_bank_config(bank_id="user-123")
print("Effective LLM provider:", config["llm_provider"])
print("Effective model:", config["llm_model"])
print("Retain model:", config.get("retain_llm_model"))

Internally, ConfigResolver.get_bank_config() performs four steps:

1. Resolves the full configuration through the global → tenant → bank hierarchy.
2. Filters to configurable fields only (as defined in config.py).
3. Strips credential fields (defined in _CREDENTIAL_FIELDS in config.py).
4. Applies any permission filters supplied by the tenant extension.

Key Source Files and Implementation Details

Understanding the following source files helps when debugging or extending the configuration system:

• hindsight-api-slim/hindsight_api/config.py (lines 81-115): Contains HindsightConfig dataclass definitions, normalize_config_key() and normalize_config_dict() helpers, and the authoritative lists of _CONFIGURABLE_FIELDS and _CREDENTIAL_FIELDS.

• hindsight-api-slim/hindsight_api/config_resolver.py (lines 43-89, 91-138): Implements the ConfigResolver class with methods resolve_full_config(), get_bank_config(), _load_bank_config(), and update_bank_config().

• hindsight-clients/python/hindsight_client_api/models/bank_config_update.py (lines 25-31): Defines the BankConfigUpdate Pydantic model that validates incoming PATCH payloads.

• hindsight-api-slim/hindsight_api/api/http.py (lines 1028-1035, 3982-3988): FastAPI router definitions for the bank configuration endpoints.

Summary

• Hindsight uses a three-tier hierarchy: global (environment variables) → tenant (optional TenantExtension) → bank (banks.config JSONB column).
• Only hierarchical fields defined in HindsightConfig.get_configurable_fields() can be overridden at the tenant or bank level.
• Update per-bank LLM settings by sending a BankConfigUpdate payload to PATCH /banks/{bank_id}/config with fields like llm_provider, llm_model, or service-tier overrides.
• The system normalizes keys automatically, accepting both snake_case (llm_provider) and env-style (HINDSIGHT_API_LLM_PROVIDER) formats.
• Credential fields (API keys, URLs) are always rejected in bank-level updates and stripped from retrieval responses for security.
• Retrieve effective settings via GET /banks/{bank_id}/config to see the merged result of all three tiers without sensitive data.

Frequently Asked Questions

What LLM settings can I override at the bank level?

You can override any field classified as hierarchical in hindsight-api-slim/hindsight_api/config.py. This includes the LLM provider, model name, API keys (if permitted by deployment policy), service tiers (such as llm_openai_service_tier or llm_groq_service_tier), and per-operation overrides like retain_llm_model and reflect_llm_model. Static and credential fields cannot be overridden at the bank level.

Why does my bank configuration update return a 400 error?

The API rejects updates that include non-hierarchical fields or credential fields such as raw API keys. The BankConfigUpdate model in bank_config_update.py validates the payload and the ConfigResolver raises a ValueError if you attempt to set restricted fields. Ensure you only provide settings like llm_provider, llm_model, or service-tier configurations that appear in the configurable fields list.

How does Hindsight handle conflicting settings between global and bank levels?

Hindsight applies a deterministic merge order: global environment variables serve as the base, tenant extensions apply middle-layer overrides, and bank-specific settings take highest precedence. When you retrieve the effective configuration via get_bank_config(), you see the final merged state where bank-level values have replaced any conflicting global or tenant values.

Can I use environment variable names instead of Python field names in my API calls?

Yes. The configuration resolver includes normalization helpers in config.py (lines 81-115) that convert environment-style keys like HINDSIGHT_API_LLM_PROVIDER to Python field names like llm_provider automatically. You can use either format interchangeably in your BankConfigUpdate payloads.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}