How Open Notebook Implements Database-First Credential Provisioning

The Open Notebook key provider implements database-first credential provisioning by querying the SurrealDB Credential table for encrypted API keys before falling back to environment variables, ensuring secure runtime injection with backward compatibility.

Open Notebook manages AI provider credentials through a dedicated key-provider module that prioritizes database-stored secrets over hard-coded environment variables. This approach enables multi-account support while maintaining compatibility with existing deployments. The implementation centers on open_notebook/ai/key_provider.py, which orchestrates credential retrieval from open_notebook/domain/credential.py and injects them into the runtime environment.

Core Workflow and Architecture

The database-first strategy follows a strict hierarchy: database records take precedence, with environment variables serving only as a fallback mechanism. This design ensures that sensitive credentials remain encrypted at rest while remaining accessible to underlying AI SDKs.

Provider-to-Environment Mapping

The system maintains a PROVIDER_CONFIG dictionary that maps simple provider names to their expected environment variable names. Located in open_notebook/ai/key_provider.py (lines 28-63), this mapping ensures the correct SDK variables are populated regardless of the source.

# Example mapping from PROVIDER_CONFIG

{
    "openai": "OPENAI_API_KEY",
    "anthropic": "ANTHROPIC_API_KEY",
    "ollama": "OLLAMA_API_KEY"
}

When provisioning occurs, the key provider uses this mapping to write database credentials into the specific environment variables that libraries like OpenAI or Anthropic expect.

Credential Retrieval Flow

The _get_default_credential(provider) function (lines 76-84) queries the credential table via Credential.get_by_provider(), returning the first matching record. The get_api_key(provider) function (lines 87-106) then implements the database-first logic:

1. Check if a database credential exists (cred.api_key)
2. If present, decrypt the stored secret using cred.api_key.get_secret_value()
3. If no database record exists, read the mapped environment variable from PROVIDER_CONFIG

This ensures that database entries always override local environment settings, allowing administrators to manage keys centrally without modifying deployment configurations.

Provider-Specific Provisioning Workflows

The system distinguishes between simple providers (single API key) and complex providers (multi-field configurations requiring specific environment variable sets).

Simple Provider Provisioning

The _provision_simple_provider(provider) method (lines 113-141) handles standard API-key-only services. It loads the credential via _get_default_credential(), writes the API key to the appropriate environment variable, and optionally sets a base_url if the credential includes one (commonly used for Ollama deployments).

from open_notebook.ai.key_provider import provision_provider_keys

async def load_openai_model():
    # Load DB-stored OpenAI key into the environment

    await provision_provider_keys("openai")
    # Now the OpenAI SDK will pick it up automatically

    return await Model.create(name="gpt-4", provider="openai")

Complex Provider Configurations

For providers requiring multiple configuration parameters, dedicated provisioning functions handle the specific environment variable sets:

Vertex AI – The _provision_vertex() function (lines 145-170) extracts project, location, and credentials_path from a vertex credential type, setting VERTEX_PROJECT, VERTEX_LOCATION, and GOOGLE_APPLICATION_CREDENTIALS.

Azure OpenAI – The _provision_azure() function (lines 174-208) writes AZURE_OPENAI_API_KEY, AZURE_OPENAI_API_VERSION, AZURE_OPENAI_ENDPOINT, and endpoint-specific variables from an azure credential record.

OpenAI-Compatible Services – The _provision_openai_compatible() function (lines 221-242) handles custom endpoints by writing OPENAI_COMPATIBLE_API_KEY and OPENAI_COMPATIBLE_BASE_URL from openai_compatible credentials.

from open_notebook.ai.key_provider import provision_provider_keys

async def use_azure_llm():
    success = await provision_provider_keys("azure")
    if not success:
        # No DB record – rely on env vars set externally

        print("Using Azure keys from environment")
    # Continue with Azure client creation …

Public API and Entry Points

Single Provider Provisioning

The provision_provider_keys(provider) function (lines 246-281) serves as the primary entry point. It normalizes the provider name, dispatches to the appropriate helper (simple or complex), and returns a boolean indicating whether environment variables were set from the database. This return value allows calling code to determine whether database-first provisioning succeeded or if fallback mechanisms were utilized.

Bulk Provisioning

The provision_all_keys() function (lines 283-307) iterates over every entry in PROVIDER_CONFIG plus all complex provider types, calling provision_provider_keys for each. While marked as deprecated for per-request use, this method remains useful for application startup scenarios:

from open_notebook.ai.key_provider import provision_all_keys

async def startup():
    results = await provision_all_keys()
    for provider, set_flag in results.items():
        print(f"{provider}: {'DB key loaded' if set_flag else 'env var used'}")

Security Model and Credential Storage

The Credential model in open_notebook/domain/credential.py stores API keys encrypted at rest using SurrealDB. The model includes fields for api_key (encrypted), optional base_url, endpoint, and provider-specific extras like project and location for Vertex AI.

When Credential.get_by_provider() (lines 39-46) retrieves a record, it returns a fully-populated Pydantic object. The key provider then extracts the secret value through get_secret_value(), ensuring the decrypted key exists only in memory and environment variables during runtime, never in logs or persistent storage.

Summary

• Database-first hierarchy: Credentials are always fetched from SurrealDB first via Credential.get_by_provider(), falling back to environment variables only when no record exists.
• Environment injection: The key provider writes decrypted credentials to SDK-specific environment variables (e.g., OPENAI_API_KEY, AZURE_OPENAI_ENDPOINT) rather than passing them directly to clients.
• Multi-provider support: Simple providers use single-key provisioning while complex providers (Vertex, Azure) receive dedicated multi-field configuration handlers.
• Security architecture: API keys remain encrypted in the database using Pydantic secret types, with decryption occurring only during the provisioning workflow in open_notebook/ai/key_provider.py.
• Backward compatibility: The boolean return pattern from provision_provider_keys() allows applications to detect when database credentials are unavailable and gracefully fall back to pre-configured environment variables.

Frequently Asked Questions

How does the key provider handle missing database credentials?

When get_api_key() finds no record in the credential table, it automatically falls back to reading the environment variable mapped in PROVIDER_CONFIG. The provision_provider_keys() function returns False to indicate that no database credentials were found, allowing application code to distinguish between database-provisioned and environment-provisioned keys.

What encryption method protects API keys in the database?

The Credential model in open_notebook/domain/credential.py uses Pydantic's SecretStr type for the api_key field, which ensures the value remains encrypted at rest within SurrealDB. The plaintext value is only exposed when explicitly retrieved via get_secret_value() during the provisioning process, preventing accidental logging of sensitive credentials.

Can I use multiple credentials for the same provider?

Currently, _get_default_credential() selects the first (oldest) credential returned by Credential.get_by_provider(), implementing a simple default selection pattern. While the database schema supports multiple credentials per provider, the key provider uses this first-match strategy to determine which key to inject into the environment.

Why does the system use environment variables instead of direct SDK configuration?

The key provider writes to environment variables (such as OPENAI_API_KEY or VERTEX_PROJECT) because underlying AI SDKs automatically read these values during client initialization. This approach decouples credential management from application logic, allowing the same codebase to run in different environments (development, staging, production) without code changes, while keeping secrets out of version control.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}