# How Open Notebook Implements a Fernet-Encrypted, Database-First Credential System

> Learn how Open Notebook implements a Fernet-encrypted credential system. It stores SurrealDB encrypted API keys, decrypting them securely into Pydantic SecretStr for your AI applications.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: internals
- Published: 2026-06-06

---

**Open Notebook stores AI-provider credentials in SurrealDB using Fernet symmetric encryption, ensuring API keys are encrypted before persistence and only decrypted when loaded into memory as Pydantic SecretStr objects.**

The `lfnovo/open-notebook` repository employs a "database-first" architecture where the `credential` table serves as the single source of truth for AI provider configurations. This design guarantees that sensitive material never touches the disk in plain text and is only decrypted transiently during runtime operations.

## Architecture Overview

The credential system consists of three tightly-coupled components implemented across specific modules:

- **[`open_notebook/domain/credential.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py)** – The domain model handling schema definition, database mapping, and encryption hooks.
- **[`open_notebook/utils/encryption.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/encryption.py)** – Fernet-based symmetric encryption utilities with lazy key loading.
- **[`open_notebook/database/repository.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/database/repository.py)** – Generic async query wrappers (`repo_query`, `ensure_record_id`) that execute SurrealQL.

This tripartite structure separates cryptographic concerns from business logic while maintaining the database-first constraint: credentials are persisted before use, and encryption/decryption occurs at the model layer, not the database layer.

## The Credential Domain Model

Located in [`open_notebook/domain/credential.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py), the `Credential` class inherits from `ObjectModel` (providing generic CRUD methods) and manages the lifecycle of encrypted API keys.

### Encryption on Save with `_prepare_save_data`

When `await cred.save()` is invoked, the model intercepts the payload through `_prepare_save_data`. This method encrypts the `api_key` (stored internally as `Optional[SecretStr]`) using `encrypt_value` before the record is sent to SurrealDB via `repo_query`. After persistence, the original `SecretStr` is restored to the instance so callers retain access to the plain key in memory while the database retains only the Fernet token.

```python
from pydantic import SecretStr
from open_notebook.domain.credential import Credential

# Create with SecretStr to prevent accidental logging

cred = Credential(
    name="Prod-OpenAI",
    provider="openai",
    modalities=["language", "embedding"],
    api_key=SecretStr("sk-my-secret-key"),
)

# Persists encrypted data; original SecretStr remains in memory

await cred.save()

```

### Decryption on Load with `_from_db_row`

Retrieval methods—`Credential.get`, `Credential.get_by_provider`, and `Credential.get_all`—delegate to `repo_query` and then pass raw database rows to the `_from_db_row` classmethod. This method decrypts the stored string using `decrypt_value` and re-wraps the result in `SecretStr`. If decryption fails (e.g., due to a rotated encryption key), the method populates a `decryption_error` field on a placeholder credential object, allowing the UI to surface actionable warnings without crashing the application.

```python

# Retrieve and decrypt automatically

loaded = await Credential.get(cred.id)
print(loaded.api_key.get_secret_value())  # Plain text accessible only here

```

### Integration with AI Providers via `to_esperanto_config`

The model exposes `to_esperanto_config()` to translate stored fields into the configuration dictionary expected by the Esperanto AI-provider factory. This method bridges the encrypted database record with runtime LLM/embedding model initialization.

```python
config = loaded.to_esperanto_config()

# Returns dict with api_key, endpoint, etc., ready for ModelManager.create_llm()

```

## Fernet Encryption Utilities

The [`open_notebook/utils/encryption.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/encryption.py) module encapsulates all cryptographic operations using the Fernet specification (AES-128-CBC with HMAC-SHA256 authentication).

### Key Derivation Strategy

Since Fernet requires a 32-byte URL-safe base64-encoded key, the `_ensure_fernet_key` function accepts any string from the `OPEN_NOTEBOOK_ENCRYPTION_KEY` environment variable (or its Docker secret variant), hashes it with **SHA-256**, and encodes the digest to meet Fernet's constraints. This allows operators to use memorable passphrases while maintaining cryptographic strength.

```python

# Key is derived lazily on first encrypt/decrypt call

from open_notebook.utils.encryption import get_fernet

fernet = get_fernet()  # Uses OPEN_NOTEBOOK_ENCRYPTION_KEY from env

```

### Lazy Loading and Token Detection

The `get_fernet()` function implements lazy loading to prevent startup crashes when the environment variable is missing. The companion function `looks_like_fernet_token` validates token length and structure, enabling the decryption routine to distinguish between encrypted Fernet tokens and legacy plaintext values.

### Graceful Degradation for Legacy Data

The `decrypt_value` function handles three states:
1. **Valid Fernet tokens**: Decrypts and returns the UTF-8 string.
2. **Malformed tokens**: Raises `ValueError` with explicit instructions to check `OPEN_NOTEBOOK_ENCRYPTION_KEY`.
3. **Legacy plaintext**: Returns the original string unchanged, ensuring backward compatibility during migrations.

## Database-First Persistence Flow

The end-to-end flow demonstrates how the system maintains its "database-first" guarantee:

1. **Instantiation**: Create `Credential` with `api_key` as `SecretStr`.
2. **Pre-save**: `_prepare_save_data` calls `encrypt_value`, transforming the plain key into a Fernet token.
3. **Persistence**: `repo_query` executes SurrealQL to store the encrypted record in the `credential` table.
4. **Retrieval**: `Credential.get` fetches the row and passes it through `_from_db_row`, which invokes `decrypt_value`.
5. **Runtime Use**: The decrypted `SecretStr` is passed via `to_esperanto_config()` to the AI factory.

If the encryption key changes between save and load operations, the retrieval method catches the decryption failure and surfaces it through the `decryption_error` field rather than exposing stack traces or key material.

## Security Design Decisions

**Why Fernet?** The implementation chooses Fernet over raw AES because it provides authenticated encryption out of the box, preventing both unauthorized reading and tampering of stored credentials.

**Key Management**: The encryption key is never hard-coded. It must be supplied via environment variables or Docker secrets, and the SHA-256 derivation ensures compatibility with arbitrary passphrase lengths.

**Memory Safety**: By using Pydantic's `SecretStr`, the system prevents accidental logging of API keys in stack traces or application logs. The plain text exists only transiently in memory during encryption/decryption operations.

**Error Handling**: Decryption failures result in descriptive user-facing messages via the `decryption_error` field, never propagating raw cryptographic exceptions or key material.

## Summary

- Open Notebook uses SurrealDB as the source of truth for credentials, encrypting data before it reaches the database.
- The `Credential` model in [`open_notebook/domain/credential.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py) handles transparent encryption via `_prepare_save_data` and decryption via `_from_db_row`.
- [`open_notebook/utils/encryption.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/encryption.py) provides Fernet-based utilities with SHA-256 key derivation from the `OPEN_NOTEBOOK_ENCRYPTION_KEY` environment variable.
- Pydantic `SecretStr` ensures API keys are never accidentally logged, while lazy loading prevents startup failures if encryption keys are missing.
- The system gracefully handles legacy plaintext values and provides clear error messages when decryption fails due to key rotation.

## Frequently Asked Questions

### How does Open Notebook handle credential encryption key rotation?

If the `OPEN_NOTEBOOK_ENCRYPTION_KEY` environment variable changes after credentials are saved, subsequent calls to `Credential.get()` will fail to decrypt existing Fernet tokens. The `_from_db_row` method catches these failures and populates a `decryption_error` field on the credential object, allowing administrators to identify which records need to be re-saved with the new key through the UI without crashing the application.

### Why does the implementation use SHA-256 to derive the Fernet key?

Fernet requires a 32-byte URL-safe base64-encoded key, but memorably passphrases rarely meet this format requirement. The `_ensure_fernet_key` function in [`open_notebook/utils/encryption.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/encryption.py) hashes the environment variable value with SHA-256 and base64-encodes the digest, producing a valid Fernet key from any arbitrary string while maintaining deterministic derivation (the same passphrase always produces the same key).

### What happens if the OPEN_NOTEBOOK_ENCRYPTION_KEY environment variable is missing?

The `get_fernet()` function uses lazy loading: it only attempts to read the environment variable when encryption or decryption is actually attempted. This prevents startup crashes. However, any attempt to save or load credentials will raise an error at that specific point, ensuring the application can boot in environments where credentials are not immediately needed (e.g., read-only viewer modes).

### How does the system distinguish between encrypted and legacy plaintext credentials?

The `decrypt_value` function in [`open_notebook/utils/encryption.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/encryption.py) utilizes `looks_like_fernet_token` to check if the stored value matches Fernet's structure (URL-safe base64 with specific length and padding). If the token does not look like Fernet, it is returned as-is, ensuring backward compatibility with unencrypted legacy data while new entries are automatically encrypted upon saving.