# How the Open Notebook key_provider Module Handles API Key Retrieval with Fallback Mechanisms

> Discover how Open Notebook's key_provider module retrieves API keys from SurrealDB, with automatic fallback to environment variables for seamless credential management.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: internals
- Published: 2026-06-14

---

**The Open Notebook `key_provider` module implements an asynchronous, database-first lookup strategy that retrieves API keys from SurrealDB `Credential` records and automatically falls back to environment variables when no database entry exists.**

The `key_provider` module in the `lfnovo/open-notebook` repository serves as the central authority for secure API key management, offering a unified interface that prioritizes encrypted database storage while maintaining backward compatibility with traditional environment-based configuration. This design enables seamless migration from `.env` files to database-backed credential storage without breaking existing integrations with AI provider SDKs.

## Understanding the Key Provider Architecture

The module establishes a deterministic fallback order for API key retrieval: **Database Credential → Environment Variable → None**. This hierarchy ensures that user-configured credentials in SurrealDB take precedence over system environment variables, while still allowing legacy deployments to function without database connectivity.

At the core of this system lies the `PROVIDER_CONFIG` dictionary (defined in [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py), lines 28-73), which maps simple provider names to their standard environment variable names. This single source of truth enables consistent key lookup across OpenAI, Anthropic, Google, and other supported providers.

## Database-First Lookup Strategy

### The PROVIDER_CONFIG Mapping

The `PROVIDER_CONFIG` dictionary defines the canonical environment variable names for each supported provider. For example, the "openai" provider maps to `OPENAI_API_KEY`, while "anthropic" maps to `ANTHROPIC_API_KEY`. This mapping serves dual purposes: identifying which environment variable to check during fallback, and determining which variable to populate during the provisioning phase.

### _get_default_credential Implementation

The private async function `_get_default_credential()` (lines 76-84) queries the SurrealDB `Credential` model for the first record matching the requested provider. According to the source code in [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py), this function:

```python

# Simplified representation of the lookup logic

async def _get_default_credential(provider: str) -> Optional[Credential]:
    # Queries Credential table for provider-specific records

    # Returns the first matching credential or None

```

This database query represents the first step in the retrieval chain, checking for encrypted keys stored via the application's frontend interface.

## Environment Variable Fallback Mechanism

When `_get_default_credential()` returns `None` or the retrieved credential lacks an `api_key`, the `get_api_key()` function (lines 87-108) executes the fallback logic. As implemented in [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py), this public async function:

1. Calls `_get_default_credential()` to attempt database retrieval
2. Returns the secret value immediately if a valid database credential exists
3. Falls back to `os.environ.get()` using the provider's mapped environment variable name from `PROVIDER_CONFIG`
4. Returns `None` only when both sources fail to yield a key

```python

# Example: Retrieve a single key with automatic fallback

import asyncio
from open_notebook.ai.key_provider import get_api_key

async def demo():
    openai_key = await get_api_key("openai")
    if openai_key:
        print("OpenAI key loaded (source: database or environment)")
    else:
        print("No OpenAI key configured")

asyncio.run(demo())

```

## Provisioning API Keys to Environment Variables

The module extends beyond simple retrieval by providing **provisioning functions** that inject database-stored credentials into the process environment, ensuring compatibility with SDKs that expect standard environment variables.

### Simple Provider Provisioning

For providers requiring only an API key, `_provision_simple_provider()` (lines 120-141) handles the environment setup. This function:

- Retrieves the credential from SurrealDB using `_get_default_credential()`
- Sets the corresponding environment variable (e.g., `OPENAI_API_KEY`)
- Optionally propagates a stored `base_url` to `<PROVIDER>_API_BASE` when present
- Logs actions without exposing secret values

### Complex Provider Handlers

Complex providers requiring multiple configuration parameters have dedicated provisioning helpers:

- **`_provision_vertex()`**: Maps Google Vertex AI project, location, and credentials path to respective environment variables
- **`_provision_azure()`** (lines 174-203): Handles Azure OpenAI-specific fields including endpoint URLs, API versions, and deployment names
- **`_provision_openai_compatible()`**: Manages generic OpenAI-compatible endpoints with custom base URLs and key mappings

These functions ensure that sophisticated authentication schemes receive all necessary parameters from the encrypted database storage.

## Public API Methods

The module exposes two primary entry points for application integration:

**`provision_provider_keys(provider)`** (lines 146-165): Normalizes the provider name and routes to the appropriate provisioning helper. Returns `True` if any environment variable was successfully set from the database, enabling applications to verify configuration status before initializing AI clients.

**`provision_all_keys()`** (lines 182-207): Iterates over all providers defined in `PROVIDER_CONFIG` plus complex providers, calling `provision_provider_keys` for each. **Note:** This function is marked deprecated for per-request use because stale variables could persist after credential deletion, making it suitable only for application startup initialization.

```python

# Example: Provision environment variables before model instantiation

import asyncio
from open_notebook.ai.key_provider import provision_provider_keys

async def init_openai():
    # Load DB-stored key into OPENAI_API_KEY environment variable

    await provision_provider_keys("openai")
    # OpenAI SDK will now automatically detect the configured key

    print("Environment configured for OpenAI provider")

asyncio.run(init_openai())

```

```python

# Example: Bulk provisioning at application startup (use with caution)

import asyncio
from open_notebook.ai.key_provider import provision_all_keys

async def startup():
    # Deprecated for per-request use; suitable for application initialization

    results = await provision_all_keys()
    print("Provisioning summary:", results)

asyncio.run(startup())

```

## Summary

- The `key_provider` module in [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) implements a **database-first, environment-fallback** strategy for secure API key retrieval.
- **`PROVIDER_CONFIG`** (lines 28-73) serves as the single source of truth mapping providers to their standard environment variable names.
- **`get_api_key()`** (lines 87-108) queries SurrealDB `Credential` records before falling back to environment variables, returning `None` only when both sources fail.
- **Provisioning functions** (`_provision_simple_provider`, `_provision_azure`, etc.) inject database credentials into environment variables to maintain compatibility with third-party SDKs.
- **`provision_provider_keys()`** enables per-provider configuration, while **`provision_all_keys()`** (deprecated for per-request use) handles bulk initialization.
- The design supports seamless migration from `.env` files to encrypted database storage without requiring changes to downstream AI provider integrations.

## Frequently Asked Questions

### How does the key_provider module prioritize between database and environment variables?

The module always checks the SurrealDB `Credential` table first via `_get_default_credential()`. Only if this returns `None` or the credential lacks an API key does it fall back to the environment variable defined in `PROVIDER_CONFIG`. This ensures that user-configured database credentials override system environment settings.

### What happens if no API key is found in either the database or environment?

The `get_api_key()` function returns `None` when neither the database query nor the environment variable lookup yields a valid key. Applications should check for this return value and handle the missing configuration appropriately, typically by prompting the user to add credentials via the frontend interface defined in [`frontend/src/lib/api/credentials.ts`](https://github.com/lfnovo/open-notebook/blob/main/frontend/src/lib/api/credentials.ts).

### Why is provision_all_keys() marked as deprecated?

According to the source code in [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) (lines 182-207), `provision_all_keys()` is deprecated for per-request use because environment variables persist for the process lifetime. If a credential is deleted from the database after provisioning, the environment variable remains set to the old value, creating a security risk. This function remains suitable for application startup initialization but should not be called repeatedly during request processing.

### How does the module handle complex providers like Azure or Vertex AI that require multiple configuration parameters?

Complex providers use dedicated provisioning helpers: `_provision_azure()` (lines 174-203) handles endpoint URLs, API versions, and deployment configurations, while `_provision_vertex()` manages project IDs and credential paths. These functions map multiple fields from the `Credential` model to their respective environment variables, ensuring the provider's SDK receives all necessary authentication parameters.