# Database-First API Key Provisioning Pattern in Open Notebook's key_provider.py

> Discover the database-first API key provisioning pattern in Open Notebook’s key_provider.py. Securely fetch encrypted credentials from SurrealDB and integrate seamlessly.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: architecture
- Published: 2026-07-06

---

**Open Notebook implements a database-first API key provisioning pattern that retrieves encrypted credentials from SurrealDB before falling back to environment variables, dynamically injecting them into `os.environ` at runtime for seamless multi-provider AI integration.**

Open Notebook stores AI provider credentials in SurrealDB rather than relying solely on static configuration files. The [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) module implements this pattern, ensuring that secrets stored as Pydantic `SecretStr` objects are injected into the process environment only when needed, maintaining security while supporting dynamic key rotation.

## Core Components of the Provisioning System

The provisioning implementation in [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) organizes credential management around three architectural concepts: centralized configuration mapping, database-first lookup, and provider-specific environment injection.

### Provider Configuration Mapping in PROVIDER_CONFIG

The `PROVIDER_CONFIG` dictionary defined at lines 28-73 serves as the single source of truth mapping provider names to their expected environment variable names. This mapping ensures that when the system provisions keys for providers like OpenAI or Anthropic, it knows exactly which `os.environ` key to populate (e.g., `OPENAI_API_KEY` or `ANTHROPIC_API_KEY`).

### Database Lookup with _get_default_credential_

The `_get_default_credential_` function (lines 76-84) queries SurrealDB to retrieve the first credential record matching the requested provider. It returns a `Credential` Pydantic model defined in [`open_notebook/domain/credential.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py) containing the encrypted `api_key` and optional `base_url`, or `None` if no database record exists.

### Specialized Provisioning Helpers

For providers requiring simple key injection, the `_provision_simple_provider` helper (lines 113-142) handles writing the API key to the appropriate environment variable defined in `PROVIDER_CONFIG`. Complex providers require specialized handlers that manage multiple environment variables:

- **`_provision_vertex_`** (lines 145-171): Handles Google Cloud Vertex AI authentication requirements
- **`_provision_azure_`** (lines 174-189): Manages Azure OpenAI-specific variables including endpoint configuration
- **`_provision_openai_compatible_`** (lines 221-242): Configures generic OpenAI-compatible endpoints with custom base URLs

## The Database-First Execution Flow

The provisioning logic follows a strict database-first hierarchy that prioritizes runtime database records over static environment configuration.

1. **Normalization**: The provider name is lower-cased to ensure consistent lookup keys regardless of user input casing.

2. **Complex Provider Routing**: If the provider matches `"vertex"`, `"azure"`, or `"openai-compatible"`, the system dispatches to the specialized helper functions that understand the specific environment variable requirements for these multi-parameter services.

3. **Simple Provider Provisioning**: For standard providers, `_provision_simple_provider` queries the database via `_get_default_credential_`. When a record exists, it writes the API key to the environment variable specified in `PROVIDER_CONFIG` and optionally sets a `*_API_BASE` variable if the credential includes a custom base URL.

4. **Environment Fallback**: If no database record exists, `get_api_key` reads the environment variable directly while respecting the `PROVIDER_CONFIG` mapping, preserving backward compatibility for deployments using traditional environment-based configuration.

## Public API Methods

The module exposes three primary entry points for credential retrieval and provisioning.

### Retrieving Individual Keys with get_api_key

The `get_api_key` function (lines 87-110) implements the database-first lookup for single key retrieval. It attempts to fetch the credential from SurrealDB and returns the decrypted key string, falling back to the environment variable only when no database record exists.

```python
from open_notebook.ai.key_provider import get_api_key

# Returns the key string or None

openai_key = await get_api_key("openai")

```

### Provisioning Provider Environment Variables

The `provision_provider_keys` function (lines 46-81) populates `os.environ` with all required variables for a specific provider before model instantiation. This ensures downstream libraries like `openai` or `anthropic` automatically see the correct credentials without additional configuration.

```python
from open_notebook.ai.key_provider import provision_provider_keys

# Load DB-stored credentials into the process environment

await provision_provider_keys("azure")

# The Azure OpenAI client will now read AZURE_OPENAI_API_KEY, etc.

```

### Bulk Provisioning with provision_all_keys

The `provision_all_keys` function loads credentials for every configured provider. While available, this approach is deprecated for request-time use in favor of on-demand provisioning via `provision_provider_keys`.

```python
from open_notebook.ai.key_provider import provision_all_keys

# Typically called once at application startup

results = await provision_all_keys()
print(results)  # {'openai': True, 'anthropic': False, ...}

```

## Extending the Pattern for New Providers

Adding support for additional AI providers requires minimal code changes. For a provider named "mycloud" that requires only a single API key, add an entry to the `PROVIDER_CONFIG` mapping:

```python

# In open_notebook/ai/key_provider.py

PROVIDER_CONFIG["mycloud"] = {"env_var": "MYCLOUD_API_KEY"}

```

The existing `_provision_simple_provider` logic automatically handles database lookup and environment injection for any provider following this simple key pattern.

## Security and Operational Benefits

This database-first pattern provides distinct advantages over traditional environment-variable-only approaches.

**Encrypted Storage**: Credentials stored in SurrealDB use Pydantic `SecretStr` encryption, ensuring secrets never appear in code repositories or CI pipeline logs.

**Dynamic Key Rotation**: Operators can update provider credentials through the UI; subsequent requests immediately retrieve the new values without requiring application restarts or container redeployments.

**Unified Interface**: All providers share the same lookup and provisioning logic, eliminating duplicated "check database or environment" code scattered across the codebase.

## Summary

- Open Notebook's [`key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/key_provider.py) implements a database-first pattern prioritizing SurrealDB credentials over environment variables.
- The `PROVIDER_CONFIG` mapping (lines 28-73) defines the relationship between provider names and environment variable names.
- The `_get_default_credential_` function (lines 76-84) retrieves encrypted credentials from the database, returning a Pydantic `Credential` model.
- Specialized helpers like `_provision_vertex_` (lines 145-171) and `_provision_azure_` (lines 174-189) handle complex multi-variable providers.
- The `get_api_key` (lines 87-110) and `provision_provider_keys` (lines 46-81) methods provide the primary public API for credential retrieval.
- This pattern enables secure, dynamic key management without requiring application restarts when credentials rotate.

## Frequently Asked Questions

### What happens if a credential exists in both the database and environment variables?

The database-first pattern always prioritizes the SurrealDB record. When `get_api_key` or `provision_provider_keys` executes, it checks the database via `_get_default_credential_` first and only falls back to the environment variable if no database record exists. This ensures that UI-updated credentials take precedence over static configuration.

### How does Open Notebook handle providers requiring multiple configuration values?

Complex providers like Vertex AI, Azure, and OpenAI-compatible endpoints use specialized provisioning helpers. For example, `_provision_azure_` (lines 174-189) sets multiple environment variables including `AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_ENDPOINT`, and `AZURE_OPENAI_API_VERSION`, while `_provision_simple_provider` (lines 113-142) handles single-key providers automatically.

### Is provision_all_keys recommended for production use?

No, `provision_all_keys` is deprecated for request-time provisioning in production environments. Instead, use `provision_provider_keys` (lines 46-81) to load only the specific provider credentials needed for the current request, reducing memory footprint and preventing unnecessary database queries for unused providers.

### How are credentials encrypted when stored in SurrealDB?

Credentials are stored using the `Credential` Pydantic model defined in [`open_notebook/domain/credential.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py), which uses Pydantic's `SecretStr` type for the `api_key` field. This ensures the secret is encrypted at rest and only decrypted when explicitly requested through the provisioning functions, preventing accidental exposure in logs or stack traces.