# How to Migrate Provider Configurations from Environment Variables to Database in Open Notebook

> Learn to migrate Open Notebook provider configurations from environment variables to SurrealDB for secure, encrypted storage and better credential management.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: migration-guide
- Published: 2026-07-05

---

**Migrating provider configurations from environment variables to the database in Open Notebook involves moving API keys from legacy env vars into the `ProviderConfig` singleton record in SurrealDB, enabling encrypted storage and multi-credential support.**

The `lfnovo/open-notebook` project has transitioned from static environment variable configuration to a dynamic database-driven approach. This migration allows you to manage multiple API keys per provider, set defaults, and encrypt credentials at rest. The new system uses a singleton pattern in SurrealDB to ensure a single source of truth for all AI provider credentials.

## Understanding the Provider Configuration Architecture

### The Legacy Environment Variable Approach

Previously, Open Notebook relied exclusively on environment variables like `OPENAI_API_KEY` to authenticate with AI providers. While simple, this approach limited you to one credential per provider and required application restarts to rotate keys. Environment variables also lack built-in encryption and cannot support complex scenarios like different keys for development, staging, and production environments.

### The Database-First ProviderConfig Model

The modern architecture centers on the `ProviderConfig` class in [`open_notebook/domain/provider_config.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/provider_config.py). This singleton record uses `record_id = "open_notebook:provider_configs"` to guarantee exactly one configuration document exists in SurrealDB. The model stores credentials in a `credentials: Dict[str, List[ProviderCredential]]` structure, allowing multiple `ProviderCredential` objects per provider.

Each `ProviderCredential` (defined around lines 22-44) includes an `is_default` field to designate which key to use when no explicit ID is specified. The `ProviderConfig` class provides helper methods including `get_default_config`, `get_config`, `add_config`, `delete_config`, and `set_default_config` to manage these credentials programmatically.

## Migrating from Environment Variables to SurrealDB

To migrate existing provider configurations, you read the current environment variables and persist them as encrypted database records. The migration is a one-time operation after which the application reads exclusively from SurrealDB.

```python
import os
from pydantic import SecretStr
from open_notebook.domain.provider_config import ProviderConfig, ProviderCredential

async def migrate_env_to_db():
    """Migrate legacy env vars to SurrealDB singleton."""
    config = await ProviderConfig.get_instance()
    
    # Check for OpenAI key in environment

    if os.getenv("OPENAI_API_KEY"):
        openai_cred = ProviderCredential(
            id="env-migrated-1",
            name="Production OpenAI",
            provider="openai",
            is_default=True,
            api_key=SecretStr(os.getenv("OPENAI_API_KEY")),
            base_url=None,
            model="gpt-4o",
        )
        config.add_config("openai", openai_cred)
    
    # Persist to database with automatic encryption

    await config.save()

```

The `save()` method in [`open_notebook/domain/provider_config.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/provider_config.py) (lines 129-138) handles serialization and calls `encrypt_value` from [`open_notebook/utils/encryption.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/encryption.py) when `OPEN_NOTEBOOK_ENCRYPTION_KEY` is configured. This ensures API keys remain encrypted at rest without manual intervention.

## Working with ProviderConfig in Your Application

### Loading the Singleton Instance

When the application starts, [`api/main.py`](https://github.com/lfnovo/open-notebook/blob/main/api/main.py) loads the configuration via `ProviderConfig.get_instance()`. This method pulls the latest data from SurrealDB on every call, ensuring all components use current credentials without caching stale values.

```python
from open_notebook.domain.provider_config import ProviderConfig

async def initialize_providers():
    """Load provider configuration on startup."""
    config = await ProviderConfig.get_instance()
    return config

```

### Retrieving Default Credentials

Downstream services like [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) retrieve credentials through the singleton. Use `get_default_config(provider)` to fetch the active credential for a specific provider, or `get_config(provider, config_id)` to select a specific key.

```python
config = await ProviderConfig.get_instance()

# Get the default OpenAI credential

openai_cred = config.get_default_config("openai")
if openai_cred:
    api_key = openai_cred.api_key.get_secret_value()
    # Use with your AI client...

```

### Adding and Updating Credentials

You can add new credentials without restarting the application. The `add_config` method appends to the provider's credential list, while `set_default_config` updates the `is_default` flag.

```python

# Create a development credential

dev_cred = ProviderCredential(
    id="dev-1",
    name="Development OpenAI",
    provider="openai",
    is_default=False,
    api_key=SecretStr("sk-dev-key-12345"),
    base_url=None,
    model="gpt-4o-mini",
)

# Add to configuration

config.add_config("openai", dev_cred)

# Promote to default

config.set_default_config("openai", "dev-1")

# Persist changes

await config.save()

```

## Security Features and Encryption

The `ProviderConfig` model implements transparent encryption through the `_prepare_save_data` method (lines 15-19). When `OPEN_NOTEBOOK_ENCRYPTION_KEY` is set in your environment, the system automatically encrypts sensitive values before writing to SurrealDB and decrypts them on retrieval. This eliminates the security risks of storing API keys in plain text environment variables or configuration files.

The encryption layer resides in [`open_notebook/utils/encryption.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/encryption.py) and provides `encrypt_value` and `decrypt_value` functions used by the domain model. This approach keeps credentials secure while maintaining the convenience of database-driven configuration.

## Summary

- **Open Notebook** uses a **singleton `ProviderConfig`** record in SurrealDB to manage all AI provider credentials, replacing static environment variables.
- The **`ProviderCredential`** model supports **multiple keys per provider** with a default flag for automatic selection.
- **Encryption** is automatic when `OPEN_NOTEBOOK_ENCRYPTION_KEY` is configured, securing API keys at rest.
- **`ProviderConfig.get_instance()`** loads fresh data from the database on every call, ensuring no stale credentials.
- **Migration** requires a one-time script to read existing env vars and call `add_config` followed by `save()`.

## Frequently Asked Questions

### How do I migrate my existing OpenAI API key from environment variables to the database?

Read the value from `OPENAI_API_KEY`, create a `ProviderCredential` object with `SecretStr` wrapping the key, and pass it to `config.add_config("openai", credential)`. Call `await config.save()` to persist the encrypted value to SurrealDB. Once saved, you can remove the environment variable from your deployment.

### Can I use multiple API keys for the same provider simultaneously?

Yes. The `credentials` dictionary in `ProviderConfig` stores a list of `ProviderCredential` objects per provider. You can maintain separate keys for different environments or projects, then select specific credentials using `get_config(provider, config_id)` or set different defaults per deployment.

### Is the database-stored configuration secure?

Yes. When you configure `OPEN_NOTEBOOK_ENCRYPTION_KEY`, the `save()` method automatically encrypts API keys using `encrypt_value` from [`open_notebook/utils/encryption.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/encryption.py) before writing to SurrealDB. The keys remain encrypted at rest and are only decrypted when retrieved by the application.

### Do I need to restart the application after updating credentials?

No. The `ProviderConfig.get_instance()` method fetches the latest configuration from SurrealDB on every invocation. Services like [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) always receive current credentials, allowing zero-downtime key rotation and updates.