# How Open-Notebook Multi-Provider AI Integration Works Through the Esperanto Library

> Discover how Open-Notebook's Esperanto library enables multi-provider AI integration. Learn about its three-layer architecture for abstracting LLMs, embeddings, and speech models efficiently.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: deep-dive
- Published: 2026-06-18

---

**Open-Notebook abstracts every AI service—including LLMs, embeddings, and speech models—behind the Esperanto library using a three-layer architecture that maps database credentials to environment variables, instantiates models via a centralized factory, and dynamically discovers provider capabilities.**

The `lfnovo/open-notebook` repository implements a robust multi-provider AI integration that treats OpenAI, Anthropic, Google, and Ollama as interchangeable backends. By leveraging the **Esperanto library** as a unified abstraction layer, the system decouples provider-specific implementation details from downstream features like graph workflows and chat interfaces. This architecture stores all credentials and model metadata in the database, enabling dynamic provider switching without code changes or manual environment configuration.

## The Three-Layer Integration Architecture

Open-Notebook’s multi-provider AI integration consists of three tightly-coupled modules that handle authentication, model instantiation, and provider discovery.

### Credential Storage and Environment Provisioning

The [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) module handles the translation between database-stored credentials and the environment variables expected by Esperanto. The `provision_provider_keys` function reads `Credential` records from the database and sets the appropriate environment variables (such as `OPENAI_API_KEY` or `ANTHROPIC_API_KEY`) for the current process【/open_notebook/ai/key_provider.py#L20-L80】. If no credential exists in the database, the system falls back to any variables already defined in the process environment, ensuring that Esperanto can locate required keys without manual export.

This design centralizes secret management within the `Credential` entity defined in [`open_notebook/domain/credential.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py), which stores API keys, base URLs, and provider-specific configuration options【/open_notebook/domain/credential.py#L80-L107】.

### Model Representation and Factory Creation

The [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py) file defines the `Model` record that stores the model name, provider, type, and an optional link to a `Credential`. The `ModelManager` class serves as the primary interface for model instantiation, loading the `Model` record from the database, resolving the associated credential (or calling `provision_provider_keys` to set environment variables), normalizing the provider name, and finally creating the concrete Esperanto object【/open_notebook/ai/models.py#L19-L75】【/open_notebook/ai/models.py#L98-L176】.

The `ModelManager` uses Esperanto’s `AIFactory` to create provider-agnostic instances:

- `AIFactory.create_language()` for chat and completion models
- `AIFactory.create_embedding()` for vector models
- `AIFactory.create_speech_to_text()` and `AIFactory.create_text_to_speech()` for audio models

The resulting objects conform to common interfaces (`LanguageModel`, `EmbeddingModel`, `SpeechToTextModel`, `TextToSpeechModel`), making the provider completely interchangeable for downstream code.

### Provider Discovery and Registration

The [`open_notebook/ai/model_discovery.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/model_discovery.py) module automates the population of available models by walking the public APIs of every supported provider. Discovery functions query OpenAI, Anthropic, Google, Ollama, and other backends to build lists of `DiscoveredModel` objects, using the environment variables that `key_provider` populated to authenticate requests【/open_notebook/ai/model_discovery.py#L1-L90】【/open_notebook/ai/model_discovery.py#L98-L150】.

This ensures that the same credential source used for runtime model creation is also used for discovery, allowing the system to automatically register new models immediately after a credential is added.

## Runtime Execution Flow

When a user selects a model in the UI, the integration executes a predictable sequence:

1. The UI sends the model ID to the backend API.
2. The API calls `ModelManager.get_model`, which loads the `Model` record and invokes `provision_provider_keys` if a credential is attached.
3. The manager builds the appropriate Esperanto model object using the factory methods.
4. Downstream code receives an abstract interface (e.g., `LanguageModel`) and executes provider-agnostic operations like `.chat()` or `.embed()`.

Because all provider data lives in the database, Open-Notebook can switch providers instantly by changing a `Model` record’s `provider` field, or re-discover models after rotating API keys.

## Implementation Examples

The following examples demonstrate how to interact with the multi-provider AI integration programmatically.

### Instantiating a Language Model

To retrieve a configured LLM for chat operations:

```python
from open_notebook.ai.models import model_manager

async def get_chat_llm(model_id: str):
    # Returns an Esperanto LanguageModel (e.g., OpenAI GPT-4)

    llm = await model_manager.get_model(model_id, temperature=0.7)
    return llm

```

### Provisioning Provider Credentials

To ensure environment variables are set before direct Esperanto usage:

```python
from open_notebook.ai.key_provider import provision_provider_keys

async def prepare_openai():
    # Loads the stored credential and sets OPENAI_API_KEY env var

    await provision_provider_keys("openai")
    # Now any Esperanto call for provider="openai" will succeed

```

### Discovering and Registering Models

To refresh the available model list across all configured providers:

```python
from open_notebook.ai.model_discovery import sync_all_providers

async def refresh_all_models():
    # Returns a dict: {provider: (discovered, new, existing)}

    stats = await sync_all_providers()
    print(stats)

```

## Summary

- **Centralized credential management**: The [`key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/key_provider.py) module maps database `Credential` records to Esperanto environment variables, eliminating manual API key configuration【/open_notebook/ai/key_provider.py#L20-L80】.
- **Factory-based model creation**: The `ModelManager` class in [`models.py`](https://github.com/lfnovo/open-notebook/blob/main/models.py) resolves database records and instantiates provider-specific objects through `AIFactory` methods, returning abstract interfaces that hide implementation details【/open_notebook/ai/models.py#L98-L176】.
- **Dynamic provider discovery**: The [`model_discovery.py`](https://github.com/lfnovo/open-notebook/blob/main/model_discovery.py) module queries provider APIs using the same credential source to populate the database with available models automatically【/open_notebook/ai/model_discovery.py#L1-L90】.
- **Runtime interchangeability**: Downstream features interact with `LanguageModel`, `EmbeddingModel`, and related abstractions, enabling seamless switching between OpenAI, Anthropic, Google, and Ollama without code changes.

## Frequently Asked Questions

### How does Open-Notebook store API keys securely?

API keys are stored as `Credential` records in the database via the Pydantic model defined in [`open_notebook/domain/credential.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py)【/open_notebook/domain/credential.py#L80-L107】. The `provision_provider_keys` function in [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) reads these records and maps them to the specific environment variable names expected by the Esperanto library, such as `OPENAI_API_KEY` or `ANTHROPIC_API_KEY`【/open_notebook/ai/key_provider.py#L20-L80】.

### Can I switch between AI providers without restarting the application?

Yes. Because provider configuration and credentials reside in the database rather than static configuration files, you can change a `Model` record’s `provider` field or update credentials dynamically. The `ModelManager` resolves these changes at runtime when `get_model` is called, and the discovery module can refresh available models immediately after adding new credentials via `sync_all_providers`.

### What model types does the Esperanto integration support?

The integration supports four primary abstraction types through Esperanto: `LanguageModel` for text generation and chat, `EmbeddingModel` for vector representations, `SpeechToTextModel` for audio transcription, and `TextToSpeechModel` for voice synthesis. The `AIFactory` creates concrete implementations for each provider behind these common interfaces.

### How does the system handle providers that require different authentication methods?

The `Credential` entity in [`open_notebook/domain/credential.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py) supports provider-specific configurations including base URLs and custom headers, while [`key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/key_provider.py) normalizes these into the standard environment variables expected by Esperanto【/open_notebook/domain/credential.py#L80-L107】. This allows the system to accommodate variations between OpenAI, Anthropic, Google, and Ollama authentication schemes within a unified framework.