How Open Notebook Implements Credential Encryption and Storage in the Database

Open Notebook uses Fernet symmetric encryption to protect API keys at rest, storing them in SurrealDB while transparently decrypting them back to Pydantic SecretStr objects on retrieval.

Open Notebook handles sensitive AI provider credentials through a transparent encryption layer that secures API keys while maintaining queryable metadata. The system stores each credential as a distinct record in the database, applying field-level encryption to ensure raw secrets never persist in clear text. This implementation leverages Python's cryptography library and Pydantic's SecretStr type to create a secure, developer-friendly credential management subsystem.

The Credential Domain Model

The Credential class in open_notebook/domain/credential.py serves as the domain model for all provider credentials. It inherits from ObjectModel, which provides standard CRUD methods like save(), get(), and get_all() that wrap SurrealDB queries.

The model uses Pydantic's SecretStr type for the api_key field. This ensures that the raw API key is never accidentally logged or serialized to JSON in plain text. When you instantiate a credential, the secret remains encapsulated:

from open_notebook.domain.credential import Credential
from pydantic import SecretStr

cred = Credential(
    name="Prod-OpenAI",
    provider="openai",
    modalities=["language", "embedding"],
    api_key=SecretStr("sk-my-very-secret-key"),
    num_ctx=16384
)

Field-Level Encryption at Rest

When save() is called, the _prepare_save_data() method (lines 27-44) intercepts the api_key value before it reaches the database. The encryption implementation resides in open_notebook/utils/encryption.py, which uses Fernet symmetric encryption from the cryptography library.

The encryption process follows these steps:

1. Key Derivation: The system reads the OPEN_NOTEBOOK_ENCRYPTION_KEY environment variable (or its Docker secret variant) and derives a valid Fernet key using SHA-256 hashing via _ensure_fernet_key().
2. Encryption: The encrypt_value() function encrypts the UTF-8 string and returns a URL-safe base64 token.
3. Storage: The encrypted token is written to the database instead of the raw key.

# Manual encryption example (used internally)

from open_notebook.utils.encryption import encrypt_value

encrypted = encrypt_value("my-plain-token")

# Returns: gAAAAAB... (URL-safe base64)

Decryption on Read Operations

When loading credentials via Credential.get() or Credential.get_all(), the system automatically reverses the encryption process. The _from_db_row() method (lines 78-84) handles the decryption, returning the value as a SecretStr instance.

The get() method (lines 57-69) implements graceful error handling: if decryption fails due to a missing or corrupted key, the system logs a warning and returns a placeholder credential containing an error message. This ensures that credential loading failures do not crash the entire application, allowing the system to continue operating with other valid credentials.

# Loading and decrypting credentials

from open_notebook.domain.credential import Credential

creds = await Credential.get_by_provider("openai")
for c in creds:
    # Decrypted automatically; access via get_secret_value()

    print(c.name, c.api_key.get_secret_value())

Flexible Configuration Storage

Beyond the encrypted API key, Open Notebook stores provider-specific settings (such as endpoints, project IDs, and context windows) in a flexible JSON field called config. The model mirrors top-level convenience fields into this bag during save operations via _prepare_save_data() (lines 44-57), and restores them on load via _mirror_config_to_fields() (lines 86-100).

This design allows the credential table to accommodate diverse provider requirements without schema migrations, while keeping commonly accessed fields like num_ctx accessible as first-class attributes.

Database Integration

All database queries flow through the generic repo_query helper in open_notebook/database/repository.py, which forwards async SQL commands to SurrealDB. The Credential model leverages its ObjectModel inheritance to wrap these raw queries with the encryption and decryption steps described above.

The API layer in api/routers/credentials.py exposes standard CRUD endpoints that utilize this model, ensuring that HTTP clients interact with decrypted secrets only when necessary and never transmit them in plain text over the wire.

Summary

• Field-level encryption: API keys are encrypted using Fernet symmetric encryption before storage in SurrealDB, implemented in open_notebook/utils/encryption.py.
• Transparent decryption: The Credential domain model automatically decrypts values when loading records, returning them as Pydantic SecretStr objects to prevent accidental exposure.
• Graceful degradation: Failed decryption attempts log warnings and return placeholder credentials rather than crashing the application.
• Flexible schema: Provider-specific settings are stored in a JSON config field, with automatic mirroring to convenience attributes for easy access.
• Environment-based keys: The encryption key is sourced from the OPEN_NOTEBOOK_ENCRYPTION_KEY environment variable and derived via SHA-256 hashing.

Frequently Asked Questions

How is the encryption key configured in Open Notebook?

Open Notebook reads the encryption key from the OPEN_NOTEBOOK_ENCRYPTION_KEY environment variable, or from a Docker secret if deployed in a containerized environment. The value is processed through _ensure_fernet_key() in open_notebook/utils/encryption.py, which uses SHA-256 hashing to derive a valid 32-byte Fernet key from the provided string. If this variable is not set, the encryption functions will fail to initialize, preventing unencrypted storage of sensitive credentials.

What happens if credential decryption fails?

When Credential.get() encounters a value that cannot be decrypted—either due to a missing encryption key or data corruption—the system logs a warning message and returns a placeholder credential containing an error description. This behavior, implemented in lines 57-69 of open_notebook/domain/credential.py, ensures that a single corrupted credential does not prevent the application from loading other valid credentials or from continuing operation.

Why does Open Notebook use Fernet encryption instead of hashing for API keys?

Fernet symmetric encryption is used because the application must retrieve the original API key value to authenticate with external AI providers. Hashing would be irreversible and useless for this purpose. The Fernet implementation provides authenticated encryption, ensuring that the decrypted data has not been tampered with, while the use of Pydantic's SecretStr type ensures the decrypted value remains protected from accidental logging or serialization after retrieval.

Does Open Notebook support legacy unencrypted credentials?

Yes, the decrypt_value() function in open_notebook/utils/encryption.py includes backward compatibility for legacy values. If the stored value does not appear to be a Fernet token (lacking the standard Fernet prefix), the function returns the value unchanged rather than raising an error. This allows the system to migrate existing clear-text credentials gradually without breaking functionality for legacy records.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}