How to Integrate Multiple AI Providers in Open Notebook Using Esperanto

Open Notebook leverages the Esperanto library as a unified façade to manage multiple LLM, embedding, and speech-to-text providers through a five-layer architecture that decouples credential storage from runtime model instantiation.

Integrating multiple AI providers in Open Notebook using Esperanto requires configuring credentials in the database, registering models that link to those credentials, and utilizing the provisioning layer to inject environment variables and instantiate provider-specific objects at runtime. This architecture allows you to switch between OpenAI, Azure, Vertex AI, and other compatible endpoints without changing application code.

Architecture Overview

The integration splits into five distinct layers that separate storage concerns from execution logic:

• Credential Storage (open_notebook/domain/credential.py): Stores API keys as SecretStr objects, along with optional base_url, model, and endpoint-specific settings. Supports multiple configurations per provider (e.g., dev vs. production keys) with optional encryption.

• Provider Configuration (open_notebook/domain/provider_config.py): A singleton record (ID open_notebook:provider_configs) that aggregates credentials by provider name and exposes helpers like get_default_config(provider) and add_config(provider, cred).

• Model Registry and Factory (open_notebook/ai/models.py): Maintains a catalogue of model records linking names to providers and credential IDs. Supports types "language", "embedding", "speech_to_text", and "text_to_speech" that map 1-to-1 to Esperanto's factory functions.

• Environment-Variable Provisioning (open_notebook/ai/key_provider.py): Copies credential values from the database into the process environment before model instantiation, ensuring Esperanto can read required keys.

• Smart Model Selection (open_notebook/ai/provision.py): Automatically selects appropriate models based on token count (switching to large_context defaults when content exceeds ~105,000 tokens) or explicit model_id overrides.

Step-by-Step Integration Workflow

1. Register Provider Credentials

Create a Credential record and attach it to the singleton ProviderConfig. Store API keys securely using Pydantic's SecretStr to prevent accidental leakage in logs.

from open_notebook.domain.provider_config import ProviderConfig, ProviderCredential
from pydantic import SecretStr
import uuid

# Create a new OpenAI credential

new_cred = ProviderCredential(
    id=str(uuid.uuid4()),
    name="OpenAI-Dev",
    provider="openai",
    is_default=False,
    api_key=SecretStr("sk-your-dev-key"),
    base_url=None,
    model="gpt-4o-mini"
)

# Load the singleton config and persist

config = await ProviderConfig.get_instance()
config.add_config("openai", new_cred)
await config.save()

2. Define Model Records

Register models in the Model table to link provider names with specific credentials and model types. Use the API endpoints defined in api/routers/models.py or interact directly with the domain model.

POST /api/models
{
  "name": "gpt-4o-mini",
  "provider": "openai",
  "type": "language",
  "credential": "<credential-record-id>"
}

Valid types include "language", "embedding", "speech_to_text", and "text_to_speech", which correspond to Esperanto's AIFactory.create_* methods.

3. Configure Default Models

Update the DefaultModels record to set your newly registered model as the default for specific workflow types. The ModelManager uses these defaults when model_id is not explicitly provided.

# Assuming model_id contains the UUID from the previous step

defaults = await model_manager.get_defaults()
defaults.default_chat_model = model_id
await defaults.save()

4. Execute AI Workflows

Use provision_langchain_model from open_notebook/ai/provision.py to obtain a LangChain-compatible model instance. This function handles token counting, default model resolution, and environment provisioning automatically.

from open_notebook.ai.provision import provision_langchain_model

messages = [{"role": "user", "content": "Explain Esperanto."}]

llm = await provision_langchain_model(
    content=messages,
    model_id=None,                # Uses default_chat_model

    default_type="chat",
    temperature=0.0
)

reply = await llm.ainvoke(messages)
print(reply)

The returned object is converted to a LangChain model via model.to_langchain() before invocation.

Environment Provisioning Implementation

Before Esperanto instantiates any provider, the system must populate environment variables. The provision_provider_keys() function in open_notebook/ai/key_provider.py queries the Credential table and exports values to the process environment.

await provision_provider_keys("openai")

This function normalizes provider names (converting underscores to hyphens to match Esperanto's expected naming convention) and handles complex providers like Azure or Vertex AI by injecting additional variables such as OPENAI_API_BASE, VERTEX_PROJECT, or AZURE_OPENAI_ENDPOINT. If no database credentials exist, the system falls back to system environment variables defined in PROVIDER_CONFIG.

Model Manager Factory Pattern

The ModelManager class in open_notebook/ai/models.py orchestrates the instantiation process:

1. Resolution: get_default_model("chat") retrieves the default model ID from the DefaultModels record.
2. Configuration: get_model(model_id) loads the Model record, pulls the linked Credential, and builds a configuration dictionary.
3. Normalization: Provider names undergo provider.replace("_", "-") to align database storage with Esperanto's hyphenated naming conventions.
4. Instantiation: The appropriate AIFactory.create_* method (e.g., create_language for LLMs) generates the provider-specific object.

This workflow ensures that graph nodes for chat, source ingestion, and transformations receive properly configured model instances without hardcoding provider specifics.

Summary

• Open Notebook uses Esperanto to abstract provider implementations, allowing seamless switching between OpenAI, Azure, and other compatible endpoints.
• Credentials are stored as SecretStr in open_notebook/domain/credential.py and aggregated by the ProviderConfig singleton.
• Model records in open_notebook/ai/models.py link providers to credentials and define types that map directly to Esperanto factory functions.
• The provision_provider_keys() function injects database credentials into environment variables before model instantiation.
• provision_langchain_model() automatically selects models based on token count (switching to large_context when exceeding ~105k tokens) and converts Esperanto objects to LangChain-compatible interfaces.

Frequently Asked Questions

What is Esperanto in the context of Open Notebook?

Esperanto is a unified abstraction library that provides a consistent interface over multiple AI providers including LLMs, embedding models, and speech services. Open Notebook uses Esperanto to decouple provider-specific implementation details from its graph-based workflows, allowing the same code to interact with OpenAI, Azure, Vertex AI, or OpenAI-compatible endpoints through standardized factory methods.

How does Open Notebook handle multiple credentials for the same provider?

The ProviderConfig singleton supports multiple Credential records per provider (e.g., separate keys for development and production). Each credential has a unique ID and optional is_default flag. When provisioning models, the system can reference specific credential IDs, or fall back to the default configuration retrieved via get_default_config(provider).

What happens if I don't specify a model_id when calling provision_langchain_model?

If model_id is None, the function queries the DefaultModels record to find the appropriate default based on the default_type parameter (e.g., "chat"). It also counts tokens in the content; if the input exceeds approximately 105,000 tokens, it automatically switches to the large_context default model to ensure the content fits within the model's context window.

How are environment variables provisioned for complex providers like Azure or Vertex AI?

The key_provider.py module handles provider-specific variable mapping. For complex providers, it injects additional environment variables beyond the basic API key, such as VERTEX_PROJECT, AZURE_OPENAI_ENDPOINT, or OPENAI_API_BASE. The system normalizes provider names by replacing underscores with hyphens to match Esperanto's expected environment variable naming conventions before instantiation occurs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}