# How to Integrate Multiple AI Providers in Open Notebook Using Esperanto

> Integrate multiple AI providers in Open Notebook using Esperanto's unified facade. This article explains how Esperanto decouples credentials and simplifies LLM management with a five-layer architecture.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-06-27

---

**Open Notebook leverages the Esperanto library as a unified façade to manage multiple LLM, embedding, and speech-to-text providers through a five-layer architecture that decouples credential storage from runtime model instantiation.**

Integrating multiple AI providers in Open Notebook using Esperanto requires configuring credentials in the database, registering models that link to those credentials, and utilizing the provisioning layer to inject environment variables and instantiate provider-specific objects at runtime. This architecture allows you to switch between OpenAI, Azure, Vertex AI, and other compatible endpoints without changing application code.

## Architecture Overview

The integration splits into five distinct layers that separate storage concerns from execution logic:

- **Credential Storage** ([`open_notebook/domain/credential.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py)): Stores API keys as `SecretStr` objects, along with optional `base_url`, `model`, and endpoint-specific settings. Supports multiple configurations per provider (e.g., dev vs. production keys) with optional encryption.

- **Provider Configuration** ([`open_notebook/domain/provider_config.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/provider_config.py)): A singleton record (ID `open_notebook:provider_configs`) that aggregates credentials by provider name and exposes helpers like `get_default_config(provider)` and `add_config(provider, cred)`.

- **Model Registry and Factory** ([`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py)): Maintains a catalogue of model records linking names to providers and credential IDs. Supports types `"language"`, `"embedding"`, `"speech_to_text"`, and `"text_to_speech"` that map 1-to-1 to Esperanto's factory functions.

- **Environment-Variable Provisioning** ([`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py)): Copies credential values from the database into the process environment before model instantiation, ensuring Esperanto can read required keys.

- **Smart Model Selection** ([`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py)): Automatically selects appropriate models based on token count (switching to `large_context` defaults when content exceeds ~105,000 tokens) or explicit `model_id` overrides.

## Step-by-Step Integration Workflow

### 1. Register Provider Credentials

Create a `Credential` record and attach it to the singleton `ProviderConfig`. Store API keys securely using Pydantic's `SecretStr` to prevent accidental leakage in logs.

```python
from open_notebook.domain.provider_config import ProviderConfig, ProviderCredential
from pydantic import SecretStr
import uuid

# Create a new OpenAI credential

new_cred = ProviderCredential(
    id=str(uuid.uuid4()),
    name="OpenAI-Dev",
    provider="openai",
    is_default=False,
    api_key=SecretStr("sk-your-dev-key"),
    base_url=None,
    model="gpt-4o-mini"
)

# Load the singleton config and persist

config = await ProviderConfig.get_instance()
config.add_config("openai", new_cred)
await config.save()

```

### 2. Define Model Records

Register models in the `Model` table to link provider names with specific credentials and model types. Use the API endpoints defined in [`api/routers/models.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/models.py) or interact directly with the domain model.

```json
POST /api/models
{
  "name": "gpt-4o-mini",
  "provider": "openai",
  "type": "language",
  "credential": "<credential-record-id>"
}

```

Valid types include `"language"`, `"embedding"`, `"speech_to_text"`, and `"text_to_speech"`, which correspond to Esperanto's `AIFactory.create_*` methods.

### 3. Configure Default Models

Update the `DefaultModels` record to set your newly registered model as the default for specific workflow types. The `ModelManager` uses these defaults when `model_id` is not explicitly provided.

```python

# Assuming model_id contains the UUID from the previous step

defaults = await model_manager.get_defaults()
defaults.default_chat_model = model_id
await defaults.save()

```

### 4. Execute AI Workflows

Use `provision_langchain_model` from [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) to obtain a LangChain-compatible model instance. This function handles token counting, default model resolution, and environment provisioning automatically.

```python
from open_notebook.ai.provision import provision_langchain_model

messages = [{"role": "user", "content": "Explain Esperanto."}]

llm = await provision_langchain_model(
    content=messages,
    model_id=None,                # Uses default_chat_model

    default_type="chat",
    temperature=0.0
)

reply = await llm.ainvoke(messages)
print(reply)

```

The returned object is converted to a LangChain model via `model.to_langchain()` before invocation.

## Environment Provisioning Implementation

Before Esperanto instantiates any provider, the system must populate environment variables. The `provision_provider_keys()` function in [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) queries the `Credential` table and exports values to the process environment.

```python
await provision_provider_keys("openai")

```

This function normalizes provider names (converting underscores to hyphens to match Esperanto's expected naming convention) and handles complex providers like Azure or Vertex AI by injecting additional variables such as `OPENAI_API_BASE`, `VERTEX_PROJECT`, or `AZURE_OPENAI_ENDPOINT`. If no database credentials exist, the system falls back to system environment variables defined in `PROVIDER_CONFIG`.

## Model Manager Factory Pattern

The `ModelManager` class in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py) orchestrates the instantiation process:

1. **Resolution**: `get_default_model("chat")` retrieves the default model ID from the `DefaultModels` record.
2. **Configuration**: `get_model(model_id)` loads the `Model` record, pulls the linked `Credential`, and builds a configuration dictionary.
3. **Normalization**: Provider names undergo `provider.replace("_", "-")` to align database storage with Esperanto's hyphenated naming conventions.
4. **Instantiation**: The appropriate `AIFactory.create_*` method (e.g., `create_language` for LLMs) generates the provider-specific object.

This workflow ensures that graph nodes for chat, source ingestion, and transformations receive properly configured model instances without hardcoding provider specifics.

## Summary

- **Open Notebook** uses Esperanto to abstract provider implementations, allowing seamless switching between OpenAI, Azure, and other compatible endpoints.
- Credentials are stored as `SecretStr` in [`open_notebook/domain/credential.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py) and aggregated by the `ProviderConfig` singleton.
- Model records in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py) link providers to credentials and define types that map directly to Esperanto factory functions.
- The `provision_provider_keys()` function injects database credentials into environment variables before model instantiation.
- `provision_langchain_model()` automatically selects models based on token count (switching to `large_context` when exceeding ~105k tokens) and converts Esperanto objects to LangChain-compatible interfaces.

## Frequently Asked Questions

### What is Esperanto in the context of Open Notebook?

Esperanto is a unified abstraction library that provides a consistent interface over multiple AI providers including LLMs, embedding models, and speech services. Open Notebook uses Esperanto to decouple provider-specific implementation details from its graph-based workflows, allowing the same code to interact with OpenAI, Azure, Vertex AI, or OpenAI-compatible endpoints through standardized factory methods.

### How does Open Notebook handle multiple credentials for the same provider?

The `ProviderConfig` singleton supports multiple `Credential` records per provider (e.g., separate keys for development and production). Each credential has a unique ID and optional `is_default` flag. When provisioning models, the system can reference specific credential IDs, or fall back to the default configuration retrieved via `get_default_config(provider)`.

### What happens if I don't specify a model_id when calling provision_langchain_model?

If `model_id` is `None`, the function queries the `DefaultModels` record to find the appropriate default based on the `default_type` parameter (e.g., `"chat"`). It also counts tokens in the content; if the input exceeds approximately 105,000 tokens, it automatically switches to the `large_context` default model to ensure the content fits within the model's context window.

### How are environment variables provisioned for complex providers like Azure or Vertex AI?

The [`key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/key_provider.py) module handles provider-specific variable mapping. For complex providers, it injects additional environment variables beyond the basic API key, such as `VERTEX_PROJECT`, `AZURE_OPENAI_ENDPOINT`, or `OPENAI_API_BASE`. The system normalizes provider names by replacing underscores with hyphens to match Esperanto's expected environment variable naming conventions before instantiation occurs.