How the Open Notebook Credential Management System Stores and Retrieves API Keys Securely

TLDR: Open Notebook’s credential management system automatically encrypts API keys before persisting them to SurrealDB and transparently decrypts them on retrieval, ensuring plaintext secrets never touch the database while keeping the developer experience seamless.

The lfnovo/open-notebook repository implements a defense-in-depth credential management system that protects provider API keys at rest without burdening application code. By encapsulating encryption logic inside the domain model, the system guarantees that sensitive values are transformed into ciphertext before storage and restored to SecretStr objects every time they are read.

Domain Model and Schema

In [open_notebook/domain/credential.py](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py), the Credential class inherits from the generic ObjectModel and maps to the SurrealDB table "credential". It declares fields such as name, provider, modalities, and api_key, with the latter typed to accept nullable secret values. This schema definition enforces structure while leaving room for optional endpoints and provider-specific settings.

The Credential Pydantic Model

The model lists every nullable field—including the sensitive api_key—and explicitly sets the table name so the repository layer knows where to persist records. Because the class is built on Pydantic, it gains validation and serialization for free, which the encryption routines later hook into during save and load operations.

Encryption at Rest

When a credential is saved, the api_key field is encrypted before it is written to SurrealDB. Inside [open_notebook/domain/credential.py](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py), the Credential._prepare_save_data() method extracts the raw string from the in-memory SecretStr, invokes encrypt_value() from [open_notebook/utils/encryption.py](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/encryption.py), and stores the resulting ciphertext instead of the plain key. The original SecretStr remains resident in memory only inside the active Python process.

Automatic Decryption on Retrieval

Retrieval methods reverse the process automatically so that callers never handle raw encrypted blobs.

Decrypting Single Credentials with get()

The Credential.get() class method overrides the parent implementation to read the raw database string, pass it through decrypt_value(), and wrap the result back into a SecretStr. After this step, developers can call cred.api_key.get_secret_value() and receive the clear text exactly as it was entered.

Bulk Retrieval and Failure Handling with get_all()

Credential.get_all() performs the same per-row decryption across every record returned by SurrealDB. If a decryption failure occurs—for example, when the server-side encryption key has changed—the method logs the error, assigns an "UNDECRYPTABLE" placeholder to that specific credential, and continues processing the remaining rows. This fallback behavior prevents a single bad record from breaking the entire UI workflow.

Transparent Developer Experience

Callers never see the encrypted value. After await cred.save(), the original SecretStr is restored in memory, and any subsequent access returns the clear text transparently. The encryption boundary is fully internal to the domain model, which means the rest of the codebase works with plaintext keys only while they reside in volatile memory.

Integration with the Esperanto Provider Factory

The to_esperanto_config() method on the Credential model converts the decrypted secret into the dictionary format expected by the Esperanto AI-provider factory. It pulls the plaintext key via self.api_key.get_secret_value() together with optional fields such as endpoint, guaranteeing that downstream LLM calls receive valid credentials without any additional handling. This design is implemented in [open_notebook/domain/credential.py](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py).

Practical Code Examples

The following examples demonstrate creating, saving, and retrieving a credential without exposing plaintext to the database layer.

from open_notebook.domain.credential import Credential
from pydantic import SecretStr

# 1️⃣ Create a new credential (the API key is a SecretStr, never logged in plain text)

cred = Credential(
    name="Production",
    provider="openai",
    modalities=["language", "embedding"],
    api_key=SecretStr("sk-MySuperSecretKey"),
)

# 2️⃣ Persist it – the key is encrypted before hitting SurrealDB

await cred.save()  # stores encrypted key; original SecretStr is kept in memory

# 3️⃣ Retrieve a credential by its ID (decryption happens automatically)

stored_cred = await Credential.get(cred.id)
print(stored_cred.api_key.get_secret_value())  # prints the clear-text key

# 4️⃣ Use the credential to build an Esperanto config for an LLM call

config = stored_cred.to_esperanto_config()

# config now contains: {"api_key": "sk-MySuperSecretKey", "endpoint": "...", ...}

In this flow, Credential.get and Credential.save act as the trust boundary: outside the model you work with normal Python objects, while inside the model everything is encrypted before it reaches SurrealDB.

Key Files in the Secure Credential Workflow

• [open_notebook/domain/credential.py](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py) — Defines the schema, orchestrates encryption and decryption, and exposes to_esperanto_config().
• [open_notebook/utils/encryption.py](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/encryption.py) — Implements encrypt_value() and decrypt_value() using the server-side secret key.
• [api/credentials_service.py](https://github.com/lfnovo/open-notebook/blob/main/api/credentials_service.py) — Provides HTTP endpoints for creating, listing, and deleting credentials by wrapping the domain model.
• [api/routers/credentials.py](https://github.com/lfnovo/open-notebook/blob/main/api/routers/credentials.py) — FastAPI router that forwards incoming requests to the service layer.
• [open_notebook/domain/provider_config.py](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/provider_config.py) — Demonstrates how multiple credentials can be associated with a provider and selected at runtime.

Summary

• Encryption at rest is enforced inside Credential._prepare_save_data(), which calls encrypt_value() before any write to SurrealDB.
• Automatic decryption occurs in Credential.get() and Credential.get_all(), restoring SecretStr objects so callers never handle ciphertext.
• Failure isolation during bulk reads substitutes an "UNDECRYPTABLE" placeholder when the encryption key changes, preventing total workflow failure.
• Zero-friction integration with Esperanto happens through to_esperanto_config(), which passes the decrypted key directly to the AI provider factory.
• The entire workflow lives in the domain model, keeping the API and routing layers free of encryption logic.

Frequently Asked Questions

How does Open Notebook prevent plaintext API keys from reaching the database?

The Credential model overrides its save preparation logic to intercept the api_key field and replace it with the output of encrypt_value() from open_notebook/utils/encryption.py. Only the resulting ciphertext is sent to SurrealDB, while the plaintext remains confined to the in-memory SecretStr.

What happens if the server-side encryption key is rotated?

If decrypt_value() fails during a read operation—most commonly because the encryption key has changed—the Credential.get_all() method logs the error and surfaces an "UNDECRYPTABLE" placeholder for that specific record. The rest of the credential list continues to load normally, and the UI can display a warning without crashing.

How does the Credential model integrate with external AI providers?

The to_esperanto_config() method converts the decrypted credential into the exact dictionary shape expected by the Esperanto provider factory. It extracts the plaintext API key with self.api_key.get_secret_value() and bundles it with optional endpoint or modality settings so that downstream LLM calls receive ready-to-use configuration.

Where is the encryption logic defined in the codebase?

The core routines encrypt_value() and decrypt_value() are implemented in [open_notebook/utils/encryption.py](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/encryption.py). The Credential class in [open_notebook/domain/credential.py](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py) is the sole consumer of these utilities, centralizing all cryptographic behavior inside the domain layer.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}