How the Open Notebook Credential System Stores API Keys Securely in SurrealDB

Open Notebook encrypts API keys using Fernet symmetric encryption before persisting them to SurrealDB, ensuring that plaintext secrets never touch the database and are only decrypted in memory when required by the application.

The lfnovo/open-notebook repository implements a defense-in-depth strategy for protecting sensitive AI provider credentials. By combining environment-driven key management with model-level encryption hooks, the system guarantees that API keys remain encrypted at rest while remaining accessible to the application logic. This article examines the complete flow from key provisioning to database persistence and retrieval.

The Encryption Architecture

The credential system relies on a two-layer approach: a Fernet symmetric encryption scheme for the data layer and Pydantic SecretStr wrappers for the application layer. The encryption key is derived from the OPEN_NOTEBOOK_ENCRYPTION_KEY environment variable, which undergoes a SHA-256 hash and Base64 encoding to produce a valid Fernet key. This process is implemented in open_notebook/utils/encryption.py.

When a Credential object is saved, the domain model intercepts the api_key field, encrypts its value, and stores only the ciphertext in SurrealDB. Upon retrieval, the model automatically decrypts the value, returning it as a SecretStr to prevent accidental logging or exposure.

How API Keys Are Encrypted Before Storage

Environment-Driven Key Provisioning

The system expects a master secret supplied via the OPEN_NOTEBOOK_ENCRYPTION_KEY environment variable or a Docker secret file. The first time encryption is needed, the code transforms this secret into a Fernet-compatible key using SHA-256 hashing followed by Base64 encoding. This ensures that even if the environment variable is a simple passphrase, the resulting encryption key meets Fernet's 32-byte requirement.

Field-Level Encryption in the Credential Model

Inside open_notebook/domain/credential.py (lines 200-210), the Credential class overrides _prepare_save_data() to intercept persistence operations. If the model contains an api_key, the method extracts the SecretStr value and passes it to encrypt_value():

from pydantic import SecretStr
from open_notebook.domain.credential import Credential

cred = Credential(
    name="Production OpenAI",
    provider="openai",
    modalities=["language"],
    api_key=SecretStr("sk-very-secret-key"),
)

await cred.save()  # encrypts api_key before writing to SurrealDB

The encrypt_value() function (located in open_notebook/utils/encryption.py) returns a URL-safe Base64 ciphertext string. This ciphertext is what actually gets stored in the api_key column of the SurrealDB record.

Secure Storage Flow in SurrealDB

Once the Credential model prepares the encrypted payload, it delegates to ObjectModel.save() (the generic ORM layer). Because encryption happens in _prepare_save_data() before the parent save method executes, SurrealDB receives only the ciphertext. The raw API key never appears in database logs, backups, or network traffic between the application and SurrealDB.

This design ensures that even if the database is compromised, an attacker gains only encrypted tokens without access to the OPEN_NOTEBOOK_ENCRYPTION_KEY environment variable.

Automatic Decryption on Retrieval

When fetching credentials, the class method Credential.get() (and the bulk variant get_all()) invokes _from_db_row() to reconstruct the model. This helper detects the Fernet token format, calls decrypt_value() to restore the plaintext, and wraps the result in a SecretStr object (see credential.py lines 124-140 and lines 31-42):

cred = await Credential.get("credential-id")
print(cred.api_key.get_secret_value())  # outputs: "sk-very-secret-key"

The system includes a graceful fallback mechanism: if the stored value is not a valid Fernet token (e.g., legacy plaintext or a token encrypted with a previous key), decrypt_value() returns the original value unchanged or raises a clear error if decryption fails. This allows for backward compatibility during key rotation scenarios.

API-Level Protection

Even after decryption, the API key is never exposed to HTTP clients. The FastAPI router in api/routers/credentials.py (lines 18-21) converts Credential instances to response objects that deliberately omit the api_key field. This creates a hard boundary where secrets are only available to internal service layers and never traverse the network to end users.

Summary

• Environment-based key management: The OPEN_NOTEBOOK_ENCRYPTION_KEY variable drives Fernet key generation via SHA-256 and Base64 encoding in open_notebook/utils/encryption.py.
• Pre-persistence encryption: The Credential model overrides _prepare_save_data() to encrypt api_key values before they reach SurrealDB.
• Ciphertext-only storage: SurrealDB stores only encrypted tokens; plaintext keys never touch the database.
• Automatic decryption: The _from_db_row() method decrypts values on retrieval and re-wraps them as SecretStr objects.
• Zero exposure API: HTTP endpoints explicitly exclude api_key from response schemas, preventing accidental leakage.

Frequently Asked Questions

How is the encryption key generated from the environment variable?

The application reads OPEN_NOTEBOOK_ENCRYPTION_KEY from the environment or a Docker secret file. On first use, it hashes the secret using SHA-256 and encodes the result in Base64 to produce a 32-byte Fernet key. This transformation happens in open_notebook/utils/encryption.py and ensures compatibility with Fernet's key requirements while allowing administrators to use human-readable passphrases.

What happens if the encryption key changes or is lost?

If the encryption key is rotated, existing ciphertext values remain decryptable only if the old key is preserved for decryption purposes. The decrypt_value() function raises a clear error if it encounters a Fernet token that cannot be decrypted. If the key is lost entirely, previously encrypted API keys become permanently inaccessible, though the system preserves the ciphertext in SurrealDB.

Can the API keys be read directly from the SurrealDB database?

No. The database contains only Fernet-encrypted ciphertext. Without the OPEN_NOTEBOOK_ENCRYPTION_KEY environment variable used by the application, the stored values are computationally infeasible to decrypt. This protects against database breaches, backup theft, or insider access to the SurrealDB instance.

Why does the system use Pydantic's SecretStr instead of plain strings?

SecretStr prevents accidental logging or serialization of sensitive values. When a credential is decrypted in credential.py, the plaintext is immediately wrapped in a SecretStr, ensuring that standard Python operations like print() or str() return masked values (e.g., **********). The actual secret is only accessible via the explicit .get_secret_value() method, making unintended exposure in logs or stack traces significantly less likely.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}