# How Open Notebook Implements Database-First API Key Storage and Encryption

> Learn how Open Notebook secures AI provider credentials with database-first API key storage and encryption. Discover its Fernet encryption and graceful decryption error handling.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-06-15

---

**Open Notebook stores AI provider credentials as encrypted SurrealDB records rather than environment variables, using Fernet encryption with a user-supplied key and supporting multiple credentials per provider with graceful decryption error handling.**

The open-notebook repository by lfnovo implements a database-first credential management system that moves API keys from global environment variables into encrypted SurrealDB records. This architecture enables per-user or per-project credential isolation while maintaining backward compatibility with legacy providers that still rely on `os.environ` lookups.

## The Credential Domain Model

The `Credential` class in [`open_notebook/domain/credential.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py) defines the schema for storing provider-specific settings. Unlike traditional configurations that use a singleton `ProviderConfig`, this model treats each credential as an independent database record with the table name `credential`.

The domain model tracks:
- **name**: Human-readable identifier
- **provider**: AI provider slug (e.g., "openai", "anthropic")
- **modalities**: Supported capabilities (language, embedding, image)
- **api_key**: Encrypted storage of the secret token
- **endpoints**: Custom base URLs for provider APIs

## Encryption at Rest

### Fernet Key Derivation

The system never persists API keys in plaintext. Instead, [`open_notebook/utils/encryption.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/encryption.py) provides `encrypt_value` and `decrypt_value` functions that use Fernet symmetric encryption. The encryption key derives from the `OPEN_NOTEBOOK_ENCRYPTION_KEY` environment variable, which accepts any user-supplied string and converts it into a valid Fernet key.

### Pre-Save Encryption Hooks

When a credential is saved, the `_prepare_save_data` method in [`open_notebook/domain/credential.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py) automatically encrypts the `api_key` field before generating the `INSERT` or `UPDATE` query. If the API key is null, the system stores `null` rather than an empty encrypted string.

```python
from open_notebook.domain.credential import Credential
from pydantic import SecretStr

cred = Credential(
    name="My OpenAI Key",
    provider="openai",
    modalities=["language", "embedding"],
    api_key=SecretStr("sk-REALLY-SECRET"),
)
await cred.save()  # Encryption happens automatically before database write

```

## Secure Retrieval and Decryption

### SecretStr Handling

During retrieval, `Credential.get` and `Credential.get_all` invoke `_from_db_row`, which detects encrypted strings and decrypts them using `decrypt_value`. The decrypted result wraps in a Pydantic `SecretStr` to prevent accidental exposure in logs or stack traces.

### Graceful Error Handling for Key Rotation

If decryption fails—such as when the `OPEN_NOTEBOOK_ENCRYPTION_KEY` changes—the code records a `decryption_error` attribute on the returned object and substitutes a placeholder `UNDECRYPTABLE` value. This allows the UI to surface a friendly warning without crashing the application or exposing malformed data.

```python

# Handle decryption error gracefully after key rotation

try:
    cred = await Credential.get("<old-credential-id>")
    if hasattr(cred, 'decryption_error'):
        print("Warning: Stored key cannot be decrypted")
except ValueError as e:
    print("Credential unusable:", e)

```

### Automatic Re-Hydration

The `save` method remembers the original `SecretStr` before delegating to the base `ObjectModel.save`. After the database round-trip, it restores the original `SecretStr` (or decrypts the returned string if the database echo contains encrypted data), ensuring the in-memory object remains usable for subsequent operations.

## Integration with AI Models

### Linking Credentials to Models

A `Model` record references a credential via the `credential` foreign key. The `Credential.get_linked_models` method queries the `model` table for all rows pointing to a specific credential ID, enabling cleanup validation before deletion.

When instantiating a model, `credential.to_esperanto_config()` produces a dictionary of provider-specific parameters (API key, endpoint, project ID) passed directly to Esperanto's `AIFactory` methods, bypassing environment variable lookups entirely.

```python
from open_notebook.domain.credential import Credential
from open_notebook.ai.models import Model

cred = await Credential.get("<credential-id>")
config = cred.to_esperanto_config()   # Returns {'api_key': 'sk-REALLY-SECRET', ...}

model = await Model.create(
    name="gpt-4o", 
    provider="openai", 
    credential=cred.id
)

# ModelManager receives config directly, not env vars

```

### Environment Variable Fallback

For legacy providers that still read from `os.environ`, [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) contains `provision_provider_keys`. This helper populates relevant `*_API_KEY` environment variables from matching credential records before initializing the provider, ensuring seamless operation during the transition to database-first storage.

## Migration from Legacy Configuration

Older installations used a single `ProviderConfig` singleton. The migration scripts in [`open_notebook/database/migrate.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/database/migrate.py) automatically create `Credential` records for each existing configuration and update model records to point at the new credential IDs. This preserves continuity while moving to the encrypted, database-first approach described in the migration notes.

```python

# List all credentials for a given provider

creds = await Credential.get_by_provider("anthropic")
for c in creds:
    print(c.name, c.api_key.get_secret_value() if c.api_key else "none")

```

## Summary

- **Database-first storage**: Credentials live as independent records in SurrealDB rather than global environment variables, supporting multiple keys per provider.
- **Fernet encryption**: [`open_notebook/utils/encryption.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/encryption.py) encrypts API keys at rest using a user-controlled `OPEN_NOTEBOOK_ENCRYPTION_KEY`.
- **Safe decryption**: `Credential.get` decrypts values into `SecretStr` objects and handles key rotation errors gracefully with `UNDECRYPTABLE` placeholders.
- **Model integration**: The `to_esperanto_config()` method provides direct configuration to Esperanto's `AIFactory`, while `provision_provider_keys` maintains backward compatibility with legacy environment-based providers.
- **Automatic migration**: Legacy `ProviderConfig` singletons convert automatically to encrypted `Credential` records.

## Frequently Asked Questions

### How is the encryption key configured?

Set the `OPEN_NOTEBOOK_ENCRYPTION_KEY` environment variable to any string before starting the application. The system derives a valid Fernet key from this value. Changing this key after credentials exist will trigger decryption errors until you re-encrypt the records.

### Can I store multiple API keys for the same provider?

Yes. Because credentials are database records rather than singleton configuration objects, you can create unlimited `Credential` records with the same provider slug (e.g., "openai"). Each `Model` instance references a specific credential ID, enabling per-project or per-user key isolation.

### What happens if I lose my encryption key?

Without the original `OPEN_NOTEBOOK_ENCRYPTION_KEY`, the stored API keys cannot be decrypted. The system returns `UNDECRYPTABLE` values and sets a `decryption_error` flag on the credential object, allowing you to identify which records need new API keys generated from the provider dashboard.

### How does the system handle providers that only read from environment variables?

The `provision_provider_keys` helper in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) populates environment variables from stored credentials before initializing legacy providers. This allows the database-first architecture to coexist with libraries that expect `OPENAI_API_KEY` or similar variables in `os.environ`.