How the Credential System in Open Notebook Manages AI Provider API Keys

Open Notebook's credential system encrypts AI provider API keys in SurrealDB using a project-wide secret, exposes them through SSRF-protected FastAPI endpoints that never return raw keys, and supports migration, validation, and connection testing via the service layer.

The credential system in Open Notebook unifies AI provider authentication under a single Credential domain model backed by SurrealDB. Every API key is encrypted at rest with OPEN_NOTEBOOK_ENCRYPTION_KEY, and the HTTP layer enforces encryption-key presence while returning only a has_api_key boolean to callers. All implementation logic is split across three primary files: open_notebook/domain/credential.py for data modeling and encryption, api/credentials_service.py for business rules, and api/routers/credentials.py for the FastAPI interface.

Credential System in Open Notebook: Domain Model and Encryption

In open_notebook/domain/credential.py, the Credential class extends ObjectModel and sets table_name = "credential". It declares fields including name, provider, modalities, and api_key, where api_key is typed as Optional[SecretStr] and included in nullable_fields.

When save() is invoked, the internal _prepare_save_data() method encrypts api_key via encrypt_value() defined in open_notebook/utils/encryption.py. On retrieval, Credential.get() and _from_db_row() decrypt the stored blob with decrypt_value() and re-wrap the plaintext in a SecretStr. This guarantees that SurrealDB never holds an unencrypted key.

For downstream AI consumption, to_esperanto_config() returns a plain dictionary containing "api_key", "base_url", and optional "endpoint" keys that Esperanto’s AIFactory consumes directly.

cred = Credential(..., api_key=SecretStr("sk-..."))
config = cred.to_esperanto_config()

# → {"api_key": "sk-...", "base_url": "...", "endpoint": "..."}

Relationship tracking is handled by get_linked_models(), which queries the model table for rows where credential = $cred_id. This lets the UI surface how many models depend on a given credential without exposing its value.

Credential System in Open Notebook: Service Layer and Validation

The api/credentials_service.py module orchestrates URL validation, connectivity testing, model discovery, and migration.

Provider Status and Environment Checks

get_provider_status() reports whether each provider is configured through database credentials or fallback environment variables, and confirms that OPEN_NOTEBOOK_ENCRYPTION_KEY is exported. check_env_configured(provider) inspects PROVIDER_ENV_CONFIG—a mapping of required environment variables per provider—to decide whether automatic migration is possible.

URL Validation and SSRF Protection

validate_url(url, provider) rejects schemes other than http or https and blocks link-local addresses in the 169.254.x.x range. This prevents server-side request forgery when users supply custom endpoints for providers such as Ollama or Azure.

Encryption-Key Enforcement

Before any write operation, require_encryption_key() verifies that OPEN_NOTEBOOK_ENCRYPTION_KEY exists. If the variable is missing, the function raises a ValueError, which the router converts into an HTTP 400 response via _handle_value_error. This stops accidental plain-text storage.

Connection Testing

test_credential(credential_id) assembles an Esperanto configuration from the stored record and dispatches to a provider-specific tester. For hosted providers it instantiates a lightweight model and performs a simple invoke; for Ollama and Azure it delegates to _test_ollama_connection and _test_azure_connection in open_notebook/ai/connection_tester.py. The result is a JSON object with provider, success, and message fields.

Model Discovery and Registration

• discover_with_config(provider, config) contacts the provider’s /models endpoint or returns a static list for providers that do not expose a discovery API.
• register_models(credential_id, models_data) creates Model records in SurrealDB, links them to the credential, and skips duplicates using a case-insensitive key set. It leverages classify_model_type from open_notebook/ai/model_discovery.py to tag model roles.

Migration Utilities

• migrate_from_provider_config() reads the legacy singleton ProviderConfig object, converts each entry into an encrypted Credential, and re-links any previously unassigned models.
• migrate_from_env() iterates over PROVIDER_ENV_CONFIG, creates a Credential per detected provider, encrypts it, and links orphaned models.

Both migrations require OPEN_NOTEBOOK_ENCRYPTION_KEY and log detailed progress to the application logs.

Credential System in Open Notebook: FastAPI Router and Safe Responses

The api/routers/credentials.py router exposes the credential system over HTTP while guaranteeing that raw API keys are never leaked. Request and response shapes are defined by Pydantic schemas in api/models.py, such as CreateCredentialRequest and CredentialResponse.

Credential CRUD Endpoints

• GET /credentials — list_credentials returns all credentials with linked model counts. An optional provider filter is supported.
• POST /credentials — create_credential validates URLs via validate_url(), enforces the encryption key, and stores a new encrypted credential.
• GET /credentials/{id} — get_credential returns metadata; the response field has_api_key reveals presence without exposing the value.
• PUT /credentials/{id} — update_credential allows partial updates and re-encrypts the key on save.
• DELETE /credentials/{id} — delete_credential cascades deletion to linked models or migrates them to another credential.

Testing, Discovery, and Migration Endpoints

Additional POST routes support operational workflows:

• /credentials/{id}/test — Calls the service-layer tester and returns connectivity results.
• /credentials/{id}/discover — Returns discoverable models for the credential’s provider.
• /credentials/{id}/register-models — Persists selected models and attaches them to the credential.
• /migrate-from-env — Triggers migrate_from_env().
• /migrate-from-provider-config — Triggers migrate_from_provider_config().
• GET /status — Returns global configuration status including encryption-key presence and per-provider readiness.
• GET /env-status — Shows which providers have environment variables set.

All routes catch service-layer ValueErrors and surface them as HTTP 400 responses.

Practical Usage Examples for the Credential System in Open Notebook

Creating a Credential via the API

import httpx

BASE_URL = "http://localhost:5055"

payload = {
    "name": "My OpenAI Key",
    "provider": "openai",
    "modalities": ["language", "embedding"],
    "api_key": "sk-xxxxxxxxxxxxxxxxxxxx",
}

resp = httpx.post(f"{BASE_URL}/credentials", json=payload)
print(resp.json())

# → {"id":"c1...", "name":"My OpenAI Key", "provider":"openai", "has_api_key":true, ...}

The endpoint handler in api/routers/credentials.py calls the service layer, enforces encryption, and intentionally omits the key from the serialized CredentialResponse.

Testing a Stored Credential

import httpx

cred_id = "c1..."
resp = httpx.post(f"{BASE_URL}/credentials/{cred_id}/test")
print(resp.json())

# Example output:

# {"provider":"openai","success":true,"message":"Connection successful"}

Under the hood, the request flows from test_credential in the router to svc_test_credential and finally to test_credential in api/credentials_service.py, which uses Esperanto to verify connectivity.

Discovering and Registering Models

Discovery:

import httpx

resp = httpx.post(f"{BASE_URL}/credentials/{cred_id}/discover")
data = resp.json()
for m in data["discovered"]:
    print(m["name"], m.get("description", ""))

Registration:

import httpx

models_to_register = [
    {"name": "gpt-4o-mini", "provider": "openai", "model_type": "language"},
    {"name": "text-embedding-3-large", "provider": "openai", "model_type": "embedding"},
]

payload = {"models": models_to_register}
resp = httpx.post(f"{BASE_URL}/credentials/{cred_id}/register-models", json=payload)
print(resp.json())

# → {"created":2, "existing":0}

Migrating from Environment Variables

# Requires OPEN_NOTEBOOK_ENCRYPTION_KEY to be set

curl -X POST http://localhost:5055/credentials/migrate-from-env

This invokes migrate_from_env in api/credentials_service.py, creating encrypted Credential records from the provider variables defined in PROVIDER_ENV_CONFIG.

Summary

• The credential system in Open Notebook persists AI provider secrets as encrypted Credential records in SurrealDB, using OPEN_NOTEBOOK_ENCRYPTION_KEY for at-rest protection.
• open_notebook/domain/credential.py handles domain modeling, encryption via encrypt_value() and decrypt_value(), and Esperanto-compatible config generation with to_esperanto_config().
• api/credentials_service.py implements business logic including SSRF-resistant URL validation, connection testing, model discovery and registration, and migrations from legacy or environment-based configs.
• api/routers/credentials.py exposes CRUD, test, and migration endpoints while guaranteeing raw API keys are never returned to callers; only a has_api_key boolean is exposed.

Frequently Asked Questions

How are API keys encrypted in Open Notebook?

API keys are encrypted at rest using the OPEN_NOTEBOOK_ENCRYPTION_KEY environment variable. When a Credential is saved in open_notebook/domain/credential.py, _prepare_save_data() calls encrypt_value() from open_notebook/utils/encryption.py to transform the SecretStr payload before it reaches SurrealDB. The key is only decrypted back into a SecretStr during retrieval inside Credential.get() or _from_db_row().

Can I migrate existing provider configurations to the credential system?

Yes. Open Notebook provides two migration paths in api/credentials_service.py: migrate_from_provider_config() converts legacy ProviderConfig records into encrypted Credential objects, and migrate_from_env() reads provider-specific environment variables listed in PROVIDER_ENV_CONFIG and creates credentials from them. Both migrations re-link orphaned models and require the encryption key to be set.

Why does the credential system block some custom endpoint URLs?

The router applies validate_url(url, provider) from the service layer, which rejects non-HTTP(S) schemes and blocks link-local addresses such as 169.254.x.x. This SSRF protection prevents the server from being coerced into making unauthorized requests to internal infrastructure when users configure custom endpoints for providers like Ollama or Azure.

How do I verify that a stored credential can connect to its provider?

Send a POST request to /credentials/{id}/test. The service layer’s test_credential() function builds an Esperanto config from the stored record, dispatches to a provider-specific tester such as _test_ollama_connection or _test_azure_connection from open_notebook/ai/connection_tester.py, and returns a JSON object containing provider, success, and message fields. If the credential is invalid or the provider unreachable, the response explains the failure.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}