How Open-Notebook Achieves Multi-Provider AI Provisioning with Esperanto

Open-Notebook achieves multi-provider AI provisioning with Esperanto by storing per-provider secrets in SurrealDB credentials, falling back to environment variables when needed, and delegating vendor-specific client creation to Esperanto's cached factory methods.

Open-Notebook's multi-provider AI provisioning with Esperanto lets the same workflow code call OpenAI, Anthropic, Azure, Ollama, or Vertex without vendor-specific imports. The lfnovo/open-notebook repository centralizes provider secrets in SurrealDB and normalizes them through a thin abstraction layer that returns ready-to-use language, embedding, and speech clients.

The Multi-Provider AI Provisioning Pipeline in Open-Notebook

The provisioning flow relies on three tightly-coupled components that transform a model identifier into a configured client.

Credential Storage in SurrealDB

Each provider secret is persisted as a Credential record in SurrealDB. The domain model in open_notebook/domain/credential.py defines a to_esperanto_config() method that decrypts the stored secret and returns a dictionary formatted for Esperanto's factory functions. This ensures API keys, base URLs, and endpoint fields are isolated from workflow logic and encrypted at rest.

Environment Fallback via key_provider

When a Model record has no linked credential, ModelManager.get_model() invokes provision_provider_keys() from open_notebook/ai/key_provider.py to inject stored keys into process environment variables. The mapping between provider name and environment variable lives in a single PROVIDER_CONFIG table, giving the system one source of truth for env-var names across simple key providers, Azure, Vertex, and OpenAI-compatible endpoints. According to the open-notebook source code, helper methods such as _provision_simple_provider, _provision_azure, and _provision_vertex handle the specific key shapes each vendor requires.

Model Construction with Esperanto

ModelManager.get_model() in open_notebook/ai/models.py loads the requested Model record, detects its type (language, embedding, speech-to-text, or text-to-speech), and normalizes the provider identifier by converting underscores to hyphens. It then calls the appropriate AIFactory.create_* function—such as AIFactory.create_language_model()—passing either the credential-derived config dict or the environment fallback. Esperanto caches the underlying provider-specific client instance, so repeated resolutions remain cheap.

How Different Provider Types Are Supported

Open-Notebook handles three broad provider categories through the same interface.

• Simple API-key providers — For vendors like OpenAI or Anthropic, Credential.api_key maps directly to config["api_key"], or the system falls back on standard environment variables such as OPENAI_API_KEY. The helper key_provider._provision_simple_provider manages this path.
• URL-based providers — For Ollama, Azure, or Vertex, the credential stores base_url and optional endpoint fields that become config["base_url"] and config["endpoint"]. Specialized helpers _provision_azure and _provision_vertex assemble these configurations.
• OpenAI-compatible endpoints — Any provider exposing the OpenAI REST schema is supported through _provision_openai_compatible, which combines a simple API key with an optional custom base URL.

Code Examples for Multi-Provider AI Provisioning

The following examples demonstrate how the provisioning layer is consumed in practice.

Load the default chat model without specifying a vendor:

from open_notebook.ai.models import model_manager

async def get_chat_llm():
    # Returns an instantiated Esperanto LanguageModel (e.g. gpt‑4, Claude, etc.)

    llm = await model_manager.get_default_model("chat")
    # The model is already configured with credentials or env vars

    return llm

Manually provision Azure keys before instantiating a model:

from open_notebook.ai.key_provider import provision_provider_keys
from open_notebook.ai.models import model_manager

async def use_azure():
    await provision_provider_keys("azure")          # pulls Azure keys from DB → env

    azure_llm = await model_manager.get_model(
        model_id="open_notebook:default_models:azure_chat",
        temperature=0.7,
    )
    # azure_llm is a TextGenerationModel ready for inference

    return azure_llm

Create an embedding model using a custom credential:

from open_notebook.ai.models import model_manager

async def embed_text(text: str):
    # Assume a model record links to a Credential storing the Anthropic API key

    embedder = await model_manager.get_model(
        model_id="open_notebook:embedding:anthropic",
        model_name="anthropic-embed-v1",
    )
    vectors = await embedder.embed([text])
    return vectors

Key Files in the Provisioning Pipeline

• open_notebook/domain/credential.py — Stores per-provider secrets, decrypts them at runtime, and builds Esperanto configuration dictionaries via to_esperanto_config().
• open_notebook/ai/key_provider.py — Maintains the PROVIDER_CONFIG mapping and exposes provision_provider_keys() to populate environment variables from database records.
• open_notebook/ai/models.py — Hosts ModelManager, which resolves model IDs, loads credentials, normalizes provider names, and delegates to Esperanto's AIFactory.
• api/routers/models.py — Exposes supported model types and provider name mappings to the frontend settings UI.

Summary

• Open-Notebook persists every provider secret as an encrypted Credential record in SurrealDB.
• The to_esperanto_config() method translates credentials into the dictionary format Esperanto expects.
• provision_provider_keys() in key_provider.py supplies environment-variable fallbacks through a centralized PROVIDER_CONFIG table.
• ModelManager.get_model() normalizes provider identifiers and routes to cached AIFactory.create_* methods.
• New providers require only an env-var mapping and the corresponding credential fields; the rest of the system consumes them automatically.

Frequently Asked Questions

What is Esperanto in Open-Notebook?

Esperanto is the open-source Python library that Open-Notebook uses as an abstraction layer over LLM, embedding, and speech providers. It exposes factory methods like AIFactory.create_language_model() that accept a standardized configuration dictionary and return a vendor-specific client. This design allows the rest of the application to remain entirely provider-agnostic.

How does Open-Notebook handle missing credentials at runtime?

If no credential is linked to a Model record, ModelManager.get_model() triggers provision_provider_keys() from open_notebook/ai/key_provider.py. This helper looks up stored secrets in SurrealDB and copies them into the appropriate environment variables based on the PROVIDER_CONFIG mapping, letting Esperanto pick them up during client initialization.

Can I add a new AI provider without modifying workflow code?

Yes. Adding a provider means defining its environment-variable mapping in key_provider.py and ensuring the UI captures the necessary fields in a Credential record. Because Credential.to_esperanto_config() and ModelManager.get_model() are generic, the new provider is automatically available through the same get_model() and get_default_model() calls.

Where is the provider-specific client cached?

Esperanto caches the instantiated client internally when AIFactory.create_* is called. Because ModelManager delegates creation to Esperanto and does not recreate clients on every request, repeated calls to get_model() for the same provider return the cached instance with minimal overhead.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}