How Open Notebook's Credential Management System Securely Stores and Retrieves API Keys

Open Notebook encrypts API keys before persisting them to SurrealDB and automatically decrypts them upon retrieval using a domain model that transparently wraps sensitive values in Pydantic SecretStr objects.

The lfnovo/open-notebook repository implements a defense-in-depth approach to protect provider credentials from unauthorized access. By leveraging Pydantic's SecretStr type alongside custom encryption utilities, the credential management system ensures API keys never exist in plaintext at rest while remaining seamlessly accessible to the application layer. This architecture encapsulates encryption and decryption logic within the domain model, allowing developers to work with clear-text keys in memory without exposing them in logs or database records.

Domain Model Architecture

The credential management system centers on a domain-driven design that separates sensitive data handling from business logic. The primary implementation resides in open_notebook/domain/credential.py, where the Credential class inherits from ObjectModel and defines the schema for storing provider credentials.

Schema Definition and Secret Handling

The model declares fields for name, provider, modalities, and api_key, with the latter typed as SecretStr to prevent accidental exposure in logs or tracebacks.

from open_notebook.domain.credential import Credential
from pydantic import SecretStr

# Create a credential - api_key is wrapped in SecretStr

cred = Credential(
    name="Production OpenAI",
    provider="openai",
    modalities=["language", "embedding"],
    api_key=SecretStr("sk-MySuperSecretKey")
)

Encryption at Rest

The system implements automatic encryption before writing to the database, ensuring that SurrealDB only stores ciphertext.

The Encryption Process

When Credential.save() is called, the internal _prepare_save_data() method extracts the secret value from the SecretStr object and passes it to encrypt_value() from open_notebook/utils/encryption.py. This transformation occurs before the data reaches the database driver.

# Persist the credential - encryption happens transparently inside save()

await cred.save()  # Stores encrypted ciphertext; original SecretStr kept in memory

The open_notebook/utils/encryption.py module provides the cryptographic primitives (encrypt_value and decrypt_value) that handle the server-side encryption key operations.

Secure Retrieval and Decryption

Retrieval methods automatically decrypt stored credentials and reconstruct SecretStr objects, ensuring callers never handle raw ciphertext.

Decryption Workflow

The Credential.get() method overrides the parent implementation to read the raw database string, invoke decrypt_value(), and wrap the result back into a SecretStr. This ensures that accessing cred.api_key.get_secret_value() returns the clear-text key only when explicitly requested.

# Retrieve by ID - decryption happens automatically

stored_cred = await Credential.get(cred.id)
print(stored_cred.api_key.get_secret_value())  # Access clear-text key

Bulk retrieval methods like get_all() and get_by_provider() perform the same per-row decryption, logging any failures while providing fallback behavior.

Error Handling and Key Rotation

The system includes resilience mechanisms for encryption key changes or data corruption scenarios.

Handling Decryption Failures

If decrypt_value() fails—such as when the server-side encryption key has changed—the model catches the exception and records a decryption_error. Rather than crashing the workflow, the system returns a minimal Credential object with an "UNDECRYPTABLE" placeholder, allowing the UI to surface a warning while preserving application stability.

# When decryption fails, get_all() returns a placeholder and logs the error

# This prevents the entire workflow from breaking due to one invalid key

Integration with AI Providers

The to_esperanto_config() method bridges credential storage with the Esperanto AI provider factory. It extracts the decrypted API key using self.api_key.get_secret_value() and constructs the configuration dictionary required for LLM calls.

# Convert to Esperanto configuration

config = stored_cred.to_esperanto_config()

# Returns: {"api_key": "sk-MySuperSecretKey", "endpoint": "...", "modalities": [...]}

This integration guarantees that downstream AI services receive properly authenticated requests without requiring manual decryption or additional handling.

API Layer Implementation

While the domain model handles encryption, the infrastructure layer exposes these capabilities through FastAPI endpoints. The api/credentials_service.py implements business logic that wraps the Credential domain methods, while api/routers/credentials.py defines the HTTP routes. This separation ensures encryption logic remains encapsulated in the domain layer while the API handles transport concerns.

Summary

• Automatic encryption: The Credential._prepare_save_data() method encrypts API keys using encrypt_value() before persistence to SurrealDB, ensuring no plaintext secrets exist at rest.
• Transparent decryption: Retrieval methods Credential.get(), get_all(), and get_by_provider() automatically decrypt ciphertext using decrypt_value() and wrap values in SecretStr objects.
• Fail-safe design: When decryption fails due to key rotation, the system returns "UNDECRYPTABLE" placeholders with error logging rather than crashing the application.
• Pydantic protection: Using SecretStr for the api_key field prevents accidental logging of sensitive values in application traces or error reports.
• Seamless integration: The to_esperanto_config() method extracts decrypted keys for AI provider configuration without exposing encryption complexity to callers.

Frequently Asked Questions

How does Open Notebook prevent API keys from appearing in logs?

The credential management system uses Pydantic's SecretStr type for the api_key field in open_notebook/domain/credential.py. This type masks the value when the model is serialized or logged, only revealing the plaintext when explicitly calling get_secret_value(). Combined with server-side encryption before database storage via encrypt_value(), this ensures keys never appear in logs or database queries as clear text.

What happens if the encryption key changes or is lost?

If the server-side encryption key changes, existing encrypted credentials cannot be decrypted with the new key. The get_all() method in the credential model catches decryption failures and returns an "UNDECRYPTABLE" placeholder while logging the decryption_error. This allows the application to continue operating and display warnings in the UI, giving administrators the opportunity to re-enter credentials with the new encryption key without breaking the entire workflow.

Can I use the credential system for providers other than OpenAI?

Yes. The Credential model in open_notebook/domain/credential.py supports any provider through its generic provider string field and modalities list. The to_esperanto_config() method dynamically constructs the configuration dictionary required by the Esperanto library, making it compatible with any AI provider supported by Esperanto, including Anthropic, Cohere, and custom endpoints defined in open_notebook/domain/provider_config.py.

Where is the actual encryption implemented?

The cryptographic functions are implemented in open_notebook/utils/encryption.py, which provides encrypt_value() and decrypt_value() functions. These utilities are called by the Credential class during the save operation (via _prepare_save_data()) and retrieval operations (via get(), get_all(), and get_by_provider()), keeping the encryption logic centralized and maintainable.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}