Multi-Provider AI Provisioning with Esperanto in Open Notebook

Open Notebook achieves multi-provider AI provisioning by using the Esperanto library as a unified abstraction layer, translating per-provider secrets stored in SurrealDB into configuration dictionaries that ModelManager.get_model() passes to AIFactory.create_* factory methods, enabling seamless switching between LLM, embedding, and speech providers.

The lfnovo/open-notebook repository eliminates provider-specific boilerplate by implementing a credential-driven provisioning system. By centralizing secrets in SurrealDB and normalizing them through Esperanto, the same workflow can invoke OpenAI, Anthropic, Azure, Ollama, or Vertex without changing inference logic. This design makes multi-provider AI provisioning both secure and extensible.

The Multi-Provider AI Provisioning Pipeline

The provisioning flow is implemented across three tightly coupled layers. When a user request hits the API, the system resolves a model identifier, loads or infers credentials, and returns a cached Esperanto client ready for inference.

Credential Storage and Translation

Provider secrets live in the Credential model inside SurrealDB. In open_notebook/domain/credential.py, each record stores fields such as api_key, base_url, and provider-specific endpoints. The model exposes a helper method called to_esperanto_config() that decrypts the stored secret and reshapes it into the exact dictionary structure Esperanto expects for its factory methods.

Environment Fallback via key_provider

When a Model record does not explicitly link a credential, the system falls back to process environment variables. The provision_provider_keys() function in open_notebook/ai/key_provider.py queries the database for any stored keys and injects them into the environment. A single PROVIDER_CONFIG table maps canonical provider names to their expected environment variable names, ensuring one source of truth for every integration.

Model Resolution and Esperanto Construction

The ModelManager.get_model() method in open_notebook/ai/models.py orchestrates the final step. It loads the Model record and either calls Credential.get_credential_obj() to retrieve a linked credential or falls back to environment variables via provision_provider_keys(). The manager then determines the model type—such as language, embedding, or speech—and normalizes the provider identifier by converting underscores to hyphens. It calls the appropriate AIFactory.create_* function, passing the credential-derived configuration dict. Esperanto caches the resulting provider-specific client, so repeated lookups remain cheap.

Provider Type Configuration Patterns

Open Notebook handles heterogeneous provider requirements through specialized provisioning paths in open_notebook/ai/key_provider.py:

• Simple API-key providers — For services like OpenAI or Anthropic, key_provider._provision_simple_provider() moves Credential.api_key into a config["api_key"] entry or the corresponding environment variable such as OPENAI_API_KEY.
• URL-based providers — For Ollama, Azure, and Vertex, key_provider._provision_azure() and _provision_vertex() map Credential.base_url and optional endpoint fields to config["base_url"] or config["endpoint"].
• OpenAI-compatible providers — Any provider exposing the OpenAI REST schema is handled by _provision_openai_compatible(), which accepts a simple key plus an optional custom base URL, maximizing reuse.

Practical Code Examples

The following examples show how the abstraction surfaces in application code.

Load the Default Chat Model

This snippet returns an instantiated Esperanto LanguageModel regardless of whether the backend is GPT-4, Claude, or another provider:

from open_notebook.ai.models import model_manager

async def get_chat_llm():
    # Returns an instantiated Esperanto LanguageModel (e.g. gpt-4, Claude, etc.)

    llm = await model_manager.get_default_model("chat")
    # The model is already configured with credentials or env vars

    return llm

Manually Provision a Provider’s Keys

For scenarios requiring explicit setup—such as Azure—the provision_provider_keys() utility seeds the environment before model creation:

from open_notebook.ai.key_provider import provision_provider_keys
from open_notebook.ai.models import model_manager

async def use_azure():
    await provision_provider_keys("azure")  # pulls Azure keys from DB → env

    azure_llm = await model_manager.get_model(
        model_id="open_notebook:default_models:azure_chat",
        temperature=0.7,
    )
    # azure_llm is a TextGenerationModel ready for inference

    return azure_llm

Create an Embedding Model with a Custom Credential

You can also request an embedding model by ID, letting the linked Credential record supply the API key:

from open_notebook.ai.models import model_manager

async def embed_text(text: str):
    # Assume a model record links to a Credential storing the API key

    embedder = await model_manager.get_model(
        model_id="open_notebook:embedding:anthropic",
        model_name="anthropic-embed-v1",
    )
    vectors = await embedder.embed([text])
    return vectors

Key Files in the Provisioning System

Understanding the following files is essential for extending or debugging the provisioning layer:

• open_notebook/domain/credential.py — Stores per-provider secrets, decrypts them, and builds the Esperanto configuration dictionaries via to_esperanto_config().
• open_notebook/ai/key_provider.py — Defines PROVIDER_CONFIG, maps providers to environment variables, and exposes provision_provider_keys().
• open_notebook/ai/models.py — Contains ModelManager.get_model(), which resolves model IDs, loads credentials, normalizes provider names, and invokes Esperanto's AIFactory.
• api/routers/models.py — Exposes supported model types to the frontend and maps provider names for UI-driven settings.

Summary

• Open Notebook persists provider secrets as Credential records in SurrealDB and translates them with to_esperanto_config().
• The key_provider.py module offers a centralized PROVIDER_CONFIG table and an environment-fallback mechanism through provision_provider_keys().
• ModelManager.get_model() normalizes provider identifiers and delegates to Esperanto's cached AIFactory.create_* methods for language, embedding, and speech models.
• Adding a new provider only requires updating its environment-variable mapping and UI fields, because the credential-to-config conversion lives entirely inside the Credential model.

Frequently Asked Questions

What is the role of Esperanto in Open Notebook's AI provisioning?

Esperanto acts as a thin, provider-agnostic abstraction layer. It exposes factory methods such as AIFactory.create_language_model() and caches the underlying client instances, so Open Notebook can switch between OpenAI, Anthropic, Groq, and others without altering inference code.

How does Open Notebook handle missing credentials at runtime?

If a Model record lacks an explicit credential linkage, ModelManager.get_model() automatically invokes provision_provider_keys() from open_notebook/ai/key_provider.py. This routine pushes stored database secrets into process environment variables based on the PROVIDER_CONFIG mapping, allowing the system to fall back transparently.

Why are provider identifiers normalized from underscores to hyphens?

Inside open_notebook/ai/models.py, ModelManager.get_model() replaces underscores with hyphens before calling Esperanto factories. This normalization ensures that provider names stored in the database—such as azure_openai—match the hyphenated identifiers Esperanto expects, like azure-openai.

Is it possible to add a new AI provider without changing core inference logic?

Yes. Because the provisioning pipeline is data-driven, adding a provider requires only defining its expected environment variables in key_provider.py, ensuring the UI captures the necessary fields, and storing the resulting secret in a Credential record. The existing ModelManager and to_esperanto_config() flow will route the new provider automatically.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}