How Fernet Encryption Secures API Credentials in SurrealDB: A Deep Dive into Open Notebook

Open Notebook uses Fernet symmetric encryption from Python's cryptography library to encrypt API keys before storing them in SurrealDB, ensuring that database dumps never expose plaintext secrets while maintaining deterministic key derivation from the OPEN_NOTEBOOK_ENCRYPTION_KEY environment variable.

Open Notebook is an open-source project that stores third-party API credentials—such as OpenAI and Anthropic keys—inside SurrealDB records. To prevent unauthorized access to these secrets, the application implements application-layer encryption using Fernet, ensuring that sensitive values are never written to disk in cleartext.

Key Derivation and Fernet Initialization

The encryption system centers on deriving a valid Fernet key from a user-provided passphrase. In open_notebook/utils/encryption.py, the _ensure_fernet_key function accepts the OPEN_NOTEBOOK_ENCRYPTION_KEY environment variable (or its Docker secret file variant) and processes it through SHA-256 hashing. The resulting digest is encoded as URL-safe base64 to produce the required 32-byte Fernet key.

This deterministic approach means the same passphrase always generates the identical encryption key, while the heavy cryptographic operations—specifically AES-128-CBC encryption with HMAC-SHA256 authentication—are handled internally by the Fernet specification.

Creating the Fernet Instance

The get_fernet() function initializes the cryptography.fernet.Fernet object using the derived key. This instance provides the encrypt() and decrypt() methods used throughout the credential management lifecycle.

Encrypting API Credentials Before Storage

When persisting provider configurations, the application converts SecretStr objects to plaintext and immediately encrypts them. In open_notebook/domain/provider_config.py, the encryption workflow calls encrypt_value, which internally invokes Fernet.encrypt().

The Encryption Process

The encrypt_value function takes the raw API key string, encrypts it using the initialized Fernet instance, and returns a UTF-8 encoded token safe for database storage. This ensures that the SurrealDB record contains only ciphertext, rendering the database immune to credential leaks even if the underlying storage is compromised.

Decryption and Backward Compatibility

Retrieving credentials requires reversing the encryption process. The decrypt_value function in open_notebook/utils/encryption.py attempts to decrypt stored values using Fernet.decrypt().

For operational flexibility, the implementation supports legacy unencrypted rows. If a value does not appear to be a Fernet token, it is returned unchanged. However, if the value resembles a Fernet token but decryption fails—indicating a mismatched OPEN_NOTEBOOK_ENCRYPTION_KEY—the system raises an explicit error alerting operators to the configuration mismatch.

Fernet Token Structure and Security Properties

The encrypted token stored in SurrealDB follows the Fernet specification format:

• Version byte (1 byte)
• Timestamp (8 bytes)
• Initialization Vector (16 bytes)
• Ciphertext (multiple of 16 bytes)
• HMAC (32 bytes)

This structure guarantees confidentiality through AES-128-CBC and integrity through HMAC-SHA256, providing authenticated encryption that detects any tampering with stored credentials.

Implementation Examples

The following examples demonstrate the encryption workflow in practice:

Example 1: Basic Encryption and Decryption

import os
os.environ["OPEN_NOTEBOOK_ENCRYPTION_KEY"] = "my-super-secret-passphrase"

from open_notebook.utils.encryption import encrypt_value, decrypt_value

# Encrypt before storage

raw_api_key = "sk-abc123def456"
encrypted = encrypt_value(raw_api_key)
print(f"Stored in SurrealDB: {encrypted}")

# Output: b'gAAAAABlY...'

# Decrypt when needed

decrypted = decrypt_value(encrypted)
assert decrypted == raw_api_key

Example 2: Integration with ProviderConfig

from pydantic import SecretStr
from open_notebook.domain.provider_config import ProviderConfig
from open_notebook.utils.encryption import encrypt_value

class SecureProviderConfig(ProviderConfig):
    def add_encrypted_credential(self, api_key_secret: SecretStr):
        # Extract plaintext from SecretStr and encrypt

        plain_key = api_key_secret.get_secret_value()
        encrypted_key = encrypt_value(plain_key)
        
        # Store encrypted version in credentials dict

        self.credentials["api_provider"].append({"api_key": encrypted_key})
        # Subsequent save() persists encrypted data to SurrealDB

Summary

• Deterministic key derivation: The _ensure_fernet_key function in open_notebook/utils/encryption.py derives 32-byte Fernet keys from OPEN_NOTEBOOK_ENCRYPTION_KEY using SHA-256 and base64 encoding.
• Application-layer encryption: API credentials are encrypted via encrypt_value before reaching SurrealDB, ensuring plaintext secrets never touch persistent storage.
• Authenticated encryption: Fernet provides AES-128-CBC confidentiality plus HMAC-SHA256 integrity verification through a standardized token format.
• Legacy support: The decrypt_value function handles both encrypted Fernet tokens and legacy plaintext values, raising explicit errors only when decryption is attempted with mismatched keys.
• Repository structure: Critical functions reside in open_notebook/utils/encryption.py while integration logic appears in open_notebook/domain/provider_config.py.

Frequently Asked Questions

What happens if I change the OPEN_NOTEBOOK_ENCRYPTION_KEY after storing credentials?

If you rotate the encryption key, existing encrypted credentials cannot be decrypted using the new key. The decrypt_value function will detect valid Fernet tokens but fail decryption, raising an explicit error indicating a key mismatch. You must maintain the original key to access previously stored credentials or re-encrypt all values during the rotation process.

Why does Open Notebook use Fernet instead of raw AES encryption?

Fernet provides a high-level, opinionated implementation of authenticated encryption that combines AES-128-CBC with HMAC-SHA256 and secure random IV generation. By using Fernet from the cryptography library, Open Notebook avoids common implementation pitfalls in raw AES usage while maintaining cross-platform compatibility and standardized token formats.

How does the system handle database dumps or backups?

Since encryption occurs at the application layer before data reaches SurrealDB, database dumps contain only Fernet-encrypted tokens. An attacker with access to the dump would need the original OPEN_NOTEBOOK_ENCRYPTION_KEY to recover usable API credentials, providing defense in depth against storage-layer compromises.

Where is the encryption logic located in the source code?

The core encryption utilities reside in open_notebook/utils/encryption.py, containing _ensure_fernet_key, get_fernet, encrypt_value, and decrypt_value. The domain logic that invokes these utilities during credential persistence is implemented in open_notebook/domain/provider_config.py, specifically around lines 123-128 where SecretStr values are processed before database upserts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}