How the Open Notebook Credential System Utilizes Esperanto for Provisioning API Keys from the Database

Open Notebook provisions AI models by converting stored database credentials into Esperanto-compatible configurations, using either direct credential injection or environment variable fallback.

Open Notebook is an open-source application that manages AI provider authentication through a secure database-backed credential system. The platform utilizes Esperanto as its abstraction layer for AI providers, requiring a translation layer between encrypted database records and the library's expected configuration format.

Database-First Architecture for Secure Credential Storage

Open Notebook stores each AI provider authentication secret in a Credential record within SurrealDB. According to the source code in open_notebook/domain/credential.py, these records contain fields such as api_key, base_url, endpoint, and provider-specific extras like num_ctx. All credentials remain encrypted at rest in the credential table, ensuring sensitive data never resides in plain text within the database.

The Two-Path Provisioning Strategy

The system provisions API keys through two distinct pathways depending on whether a model has an explicit credential link.

Direct Credential Injection via to_esperanto_config()

When a model's credential field points to a specific Credential record, the ModelManager bypasses environment variables entirely. In open_notebook/ai/models.py (lines 20-26), the get_model() method loads the credential record and invokes Credential.to_esperanto_config() (lines 102-136 of open_notebook/domain/credential.py). This method converts the stored fields into a plain Python dictionary that matches Esperanto's expected configuration schema.

The resulting dictionary passes directly to Esperanto's AIFactory.create_* methods, provisioning the model without any environment variable lookup.

from open_notebook.ai.models import model_manager

# The model record has `credential="cred:123"` in SurrealDB.

model = await model_manager.get_model("model:abc")  # Returns an Esperanto model

# Internally:

#   cred = await Credential.get("cred:123")

#   config = cred.to_esperanto_config()

#   AIFactory.create_language(..., config=config)

Environment Variable Fallback with provision_provider_keys()

If a model lacks a linked credential, the system falls back to the key provider helper in open_notebook/ai/key_provider.py. The provision_provider_keys() function (lines 46-81) retrieves the first available credential for the requested provider via _get_default_credential() and sets the appropriate environment variables (e.g., OPENAI_API_KEY).

The _provision_simple_provider() function (lines 13-43) handles the mapping of credential fields to environment variable names based on the PROVIDER_CONFIG dictionary. If no credential exists, the system leaves existing environment variables untouched, allowing traditional os.getenv-based authentication to function.

from open_notebook.ai.key_provider import provision_provider_keys

# Populate environment variables from the first OpenAI credential found.

await provision_provider_keys("openai")

# Esperanto can now create a model without an explicit credential:

from esperanto import AIFactory
llm = AIFactory.create_language(model_name="gpt-4", provider="openai")

Key Implementation Details

The Credential Model and Configuration Translation

The Credential class in open_notebook/domain/credential.py implements the to_esperanto_config() method to bridge the gap between database storage and Esperanto's requirements. This method extracts fields including api_key, base_url, endpoint, and provider-specific parameters, returning a configuration dictionary that AIFactory consumes directly.

ModelManager Integration

The ModelManager (open_notebook/ai/models.py) serves as the orchestration layer, always preferring explicit credential configurations over environment variables. When get_model() detects a linked credential, it immediately loads the record and converts it to Esperanto format. Only when the credential field is null does the manager invoke the key provider fallback.

Provider-Aware Key Mapping

The key provider system handles both simple providers (single API key) and complex providers (Azure, Vertex, OpenAI-compatible) through the PROVIDER_CONFIG mapping. This configuration determines which environment variables to set based on the provider type, ensuring that multi-field authentication schemes like Azure's (requiring endpoint and key) receive proper treatment.

Manual Configuration Building

You can manually construct Esperanto configurations from credentials for advanced use cases:

from open_notebook.domain.credential import Credential
from esperanto import AIFactory

cred = await Credential.get("cred:456")
esperanto_cfg = cred.to_esperanto_config()

# Use directly with AIFactory:

embedding = AIFactory.create_embedding(
    model_name="text-embedding-ada-002",
    provider="openai",
    config=esperanto_cfg,
)

Summary

• Database-First Storage: Credentials reside in SurrealDB's credential table with encryption at rest, eliminating the need for hardcoded secrets.
• Dual Provisioning Paths: The system uses direct credential injection via to_esperanto_config() when available, falling back to environment variable provisioning through provision_provider_keys() only when necessary.
• Esperanto Integration: The Credential class translates database records into Esperanto-compatible dictionaries, enabling seamless AIFactory model creation without environment pollution.
• Provider Flexibility: The key provider handles complex authentication schemes for Azure, Vertex, and OpenAI-compatible providers through configurable environment variable mapping.

Frequently Asked Questions

What is Esperanto in the context of Open Notebook?

Esperanto is the AI provider abstraction layer used by Open Notebook to standardize interactions with multiple large language model providers. It provides a unified interface through AIFactory methods while accepting provider-specific configurations, allowing Open Notebook to support OpenAI, Azure, Vertex, and other providers through a single integration point.

How does the credential system handle missing credentials?

When a model lacks a linked credential, the system invokes provision_provider_keys() from open_notebook/ai/key_provider.py. This function searches for the first available credential for the requested provider and sets the corresponding environment variables. If no credential exists in the database, the system preserves existing environment variables, allowing standard os.getenv authentication to proceed.

What database does Open Notebook use for credential storage?

Open Notebook uses SurrealDB to store credential records. The Credential domain model in open_notebook/domain/credential.py defines the schema for these records, which include fields for API keys, base URLs, endpoints, and provider-specific configuration options. All credentials are encrypted at rest within the SurrealDB credential table.

How are credentials protected at rest in the database?

Credentials stored in SurrealDB are encrypted at rest, ensuring that sensitive authentication data such as API keys remain protected even if database access is compromised. The encryption occurs at the database level, while the application layer interacts with decrypted credential objects only when building Esperanto configurations through the to_esperanto_config() method.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}