How the Database-First Credential Provisioning Pattern Works in Open-Notebook

Open-Notebook implements database-first credential provisioning by storing AI provider authentication data as encrypted, individual records in SurrealDB rather than environment variables or singleton config objects, enabling dynamic model linking and seamless migrations.

This architecture shifts the source of truth from static configuration files to queryable database entities. By treating credentials as first-class domain objects in the lfnovo/open-notebook codebase, the system supports multiple API keys per provider, granular access control, and runtime credential rotation without application restarts.

Core Architecture

The pattern centers on the Credential domain model in open_notebook/domain/credential.py, which maps directly to the credential table in SurrealDB. Unlike legacy systems that rely on a singleton ProviderConfig object or environment variable singletons, each provider account receives its own encrypted row with independent lifecycle management.

The Credential Domain Model

The Credential class extends a Pydantic base model and defines fields for api_key, base_url, project_id, and modalities. All sensitive values use Pydantic SecretStr to prevent accidental logging, while the save() method automatically encrypts the API key using AES-GCM before persistence. The encryption key derives from the OPEN_NOTEBOOK_ENCRYPTION_KEY environment variable via utilities in open_notebook/utils/encryption.py.

When retrieving records, Credential.get(id) and Credential.get_all() decrypt the stored API key and return fully hydrated objects. If decryption fails—such as when the encryption key rotates—the error is captured in a decryption_error field, allowing the UI to warn users without crashing the application.

Relationship to AI Models

Each AI model defined in open_notebook/ai/models.py maintains a foreign-key reference credential. The method cred.get_linked_models() queries the database for all models associated with a specific credential, establishing a one-to-many relationship that enables cascading updates. When you update a credential's API key, all linked models immediately use the new authentication data on their next invocation.

Service Layer Operations

Business logic lives in api/credentials_service.py, which orchestrates creation, migration, discovery, and deletion while enforcing security policies.

Migration from Environment Variables

The migrate_from_env() function scans the PROVIDER_ENV_CONFIG dictionary for providers configured via environment variables. For each detected provider, it constructs a Credential object, encrypts the API key, and persists it to the database. After migration, it re-assigns any orphaned models to the newly created credential, ensuring continuity of service.

Similarly, migrate_from_provider_config() handles the legacy singleton pattern. It reads existing ProviderConfig records, converts each stored credential into an individual Credential row, and updates model references to point to the new database-first entities.

Model Discovery and Registration

The discover_models_for_credential() workflow demonstrates the power of the database-first approach. It calls cred.to_esperanto_config() to build a plain dictionary of non-null configuration values, passes this to provider-specific discovery functions, and then uses register_models() to persist discovered models in the model table with the credential foreign key set. This links discovered capabilities directly to the authentication context that enables them.

API Security and Access Controls

The FastAPI router in api/routers/credentials.py exposes CRUD endpoints that delegate to the service layer while enforcing strict security boundaries. Endpoints never return raw API keys; instead, they expose metadata such as has_api_key (boolean) and decryption_error (string).

Security Guarantees

The implementation provides defense in depth:

• Encryption at rest: All API keys use AES-GCM encryption before database insertion
• SSRF protection: The validate_url function inspects base_url fields to block dangerous internal addresses
• Secret masking: SecretStr types ensure that even if objects are logged, sensitive values appear as ********
• Graceful degradation: Decryption failures surface as user warnings rather than runtime exceptions

Practical Implementation Examples

Create a credential programmatically and persist it to the database:

from open_notebook.domain.credential import Credential
from pydantic import SecretStr

cred = Credential(
    name="Production OpenAI Key",
    provider="openai",
    modalities=["language", "embedding"],
    api_key=SecretStr("sk-..."),
)
await cred.save()  # Encrypts and INSERTs into SurrealDB

Migrate legacy environment configuration to the database:

from api.credentials_service import migrate_from_env

result = await migrate_from_env()
print(f"Migrated {len(result['migrated'])} providers to database-first storage")

Test connectivity and discover models for an existing credential:

from api.credentials_service import test_credential, discover_with_config, register_models

# Verify the credential works

status = await test_credential("<credential-id>")

# Discover available models

cred = await Credential.get("<credential-id>")
config = cred.to_esperanto_config()
discovered = await discover_with_config(cred.provider, config)

# Register them with the foreign key relationship

await register_models(cred.id, discovered)

Handle deletion with model reassignment:

# Delete credential and migrate linked models to another

await delete_credential(
    credential_id="old-cred-id",
    migrate_to="new-cred-id"  # Optional: prevents cascade deletion

)

Summary

• Database-first architecture treats credentials as queryable SurrealDB records rather than static configuration, enabling runtime management and multi-tenancy
• Automatic encryption via encrypt_value in credential.py ensures API keys never persist in plain text
• Service layer abstraction in credentials_service.py handles complex migrations from ProviderConfig singletons and environment variables
• Model linking establishes foreign-key relationships between credentials and AI models, allowing dynamic discovery and cascading updates
• Security hardening combines AES-GCM encryption, URL validation, and SecretStr masking to protect sensitive authentication data

Frequently Asked Questions

What is database-first credential provisioning?

Database-first credential provisioning is an architectural pattern where authentication secrets are stored as individual database records rather than environment variables or configuration files. In Open-Notebook, this means each AI provider API key exists as an encrypted row in the credential table, allowing the system to query, update, and link credentials to specific models at runtime without requiring application restarts or configuration redeployments.

How does Open-Notebook encrypt stored API keys?

The system uses AES-GCM encryption implemented in open_notebook/utils/encryption.py. When Credential.save() is called, the api_key field—wrapped in a Pydantic SecretStr—is encrypted using the key derived from OPEN_NOTEBOOK_ENCRYPTION_KEY. The encrypted ciphertext is stored in SurrealDB, while the decryption only occurs when Credential.get() or get_all() retrieves the record for active use.

Can I migrate existing environment-based credentials to the database?

Yes. The migrate_from_env() function in api/credentials_service.py scans PROVIDER_ENV_CONFIG and converts environment-based credentials into database records. After creation, it automatically re-assigns any unlinked models to the new credentials. This migration path allows teams to transition from static configuration to the database-first pattern without losing existing model associations.

What happens to linked models when I delete a credential?

The deletion endpoint in api/routers/credentials.py supports a migrate_to parameter. If provided, all models referencing the deleted credential are updated to point to the specified replacement credential. If omitted, the system cascade-deletes the linked models. The implementation gracefully handles decryption errors during this process by falling back to raw database operations when necessary.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}