# How Open-Notebook Implements Multi-Provider AI Provisioning: A Deep Dive into the Esperanto Integration

> Discover how Open-Notebook achieves multi-provider AI provisioning with Esperanto. Explore its provider-agnostic layer supporting OpenAI, Anthropic, Google, and more without code modifications.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: deep-dive
- Published: 2026-07-04

---

**Open-Notebook implements multi-provider AI provisioning through a provider-agnostic layer using the Esperanto library, combining SurrealDB-stored model metadata with dynamic credential injection to support OpenAI, Anthropic, Google, and dozens of other providers without code changes.**

The `lfnovo/open-notebook` repository provides a flexible, database-driven architecture for managing AI models across disparate vendors. By abstracting provider-specific authentication and initialization behind a unified provisioning system, it enables seamless runtime switching between large language models, embedding services, and speech processors. This implementation leverages the Esperanto library to normalize client interactions while maintaining secure credential storage in SurrealDB.

## The Three-Layer Architecture for Multi-Provider AI Provisioning

The system follows a three-tier design that separates model definition, credential management, and runtime instantiation.

### SurrealDB Model Registry

In [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py), the `Model` class serves as the primary interface to the SurrealDB backend. Model records specify critical metadata including `name`, `provider`, `type` (language, embedding, speech-to-text, text-to-speech), and optional links to **Credential** records.

The `Model` class provides utility methods to retrieve these configurations:

- `Model.get(id)` – Fetches a specific model record
- `Model.get_models_by_type(type)` – Filters models by capability
- `Model.get_credential_obj()` – Retrieves associated authentication data

### Credential Injection and Environment Provisioning

When `ModelManager.get_model` processes a request, it first checks for linked credentials. If present, the system extracts configuration via `credential.to_esperanto_config()`. Otherwise, it falls back to environment variables managed by [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py).

The [`key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/key_provider.py) module maps each provider to its expected environment variable names through the `PROVIDER_CONFIG` dictionary:

- `OPENAI_API_KEY` for OpenAI
- `ANTHROPIC_API_KEY` for Anthropic
- `OLLAMA_API_BASE` for Ollama

The module exposes `provision_provider_keys()` and `provision_all_keys()` to populate `os.environ` dynamically from database records, with dedicated handlers for complex providers like Azure (`_provision_azure`) and Google Vertex (`_provision_vertex`).

### Dynamic Model Creation via Esperanto

After configuration resolution, the `ModelManager` instantiates the appropriate Esperanto client. Provider names undergo normalization (`provider.replace("_", "-")`) to match Esperanto's hyphenated format conventions.

The `AIFactory` methods create specialized clients:

- `create_language()` for chat models
- `create_embedding()` for vector models
- `create_speech_to_text()` and `create_text_to_speech()` for audio processing

Esperanto caches these objects automatically, ensuring subsequent requests reuse existing connections.

## High-Level Model Selection Logic

The `provision_langchain_model` function in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) orchestrates intelligent model selection based on request characteristics:

1. **Large Context Detection**: Content exceeding **105,000 tokens** triggers the large-context default model
2. **Explicit Model ID**: Direct loading when `model_id` is provided
3. **Default Type Fallback**: Uses the configured default for the requested operation type (chat, transformation, embeddings)

This logic integrates with the singleton `model_manager = ModelManager()` instantiated at the module level in [`models.py`](https://github.com/lfnovo/open-notebook/blob/main/models.py).

## Practical Implementation Examples

### Provisioning a Language Model for Chat

```python
from open_notebook.ai.provision import provision_langchain_model

content = "Give me a detailed summary of the PDF."

# Use default chat model unless content is huge

chat_model = await provision_langchain_model(
    content=content,
    model_id=None,          # No explicit model ID

    default_type="chat",   # Look up default_chat_model in DB

)

# `chat_model` is a LangChain `BaseChatModel` ready to be used

response = await chat_model.ainvoke({"messages": [{"role": "user", "content": content}]})

```

### Direct Model Access via Model Manager

```python
from open_notebook.ai.models import model_manager

# Assume a model record with ID "model:openai:gpt-4" exists

gpt4 = await model_manager.get_model("model:openai:gpt-4")

# The call internally:

#   - fetches the Model record → loads its Credential

#   - populates OPENAI_API_KEY via key_provider

#   - creates AIFactory.create_language(name="gpt-4", provider="openai")

```

### Environment Variable Provisioning at Startup

```python
from open_notebook.ai.key_provider import provision_all_keys

# Load all stored credentials into os.environ (run once on app start)

await provision_all_keys()

```

## Key Architectural Components

| Component | Location | Purpose |
|-----------|----------|---------|
| **Model Definition** | [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py) | `Model` and `DefaultModels` classes for metadata storage |
| **Credential Storage** | [`open_notebook/domain/credential.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py) | Database schema for API keys and secrets |
| **Environment Provisioning** | [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) | Maps providers to env-vars (`PROVIDER_CONFIG`) |
| **Selection Logic** | [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) | Token-count-driven model selection |
| **Provider Integration** | Esperanto library | Normalized client creation via `AIFactory` |

## Summary

- **Open-Notebook** achieves multi-provider AI provisioning through a database-driven architecture that separates model metadata from runtime configuration.
- The **Esperanto** library provides the abstraction layer, with `AIFactory` methods creating provider-specific clients while normalizing naming conventions.
- **Credential management** in [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) supports both database-stored secrets and environment variables, with specialized handlers for Azure and Vertex AI.
- **Intelligent selection** in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) routes requests based on token count (105,000 threshold), explicit IDs, or default model types.
- The singleton **ModelManager** caches Esperanto instances, ensuring efficient reuse across the application lifecycle.

## Frequently Asked Questions

### How does Open-Notebook handle different API key formats across providers?

Open-Notebook uses the `PROVIDER_CONFIG` mapping in [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) to standardize environment variable names like `OPENAI_API_KEY` and `ANTHROPIC_API_KEY`. For complex providers such as Azure and Google Vertex, dedicated methods `_provision_azure` and `_provision_vertex` handle multi-parameter authentication, extracting these values from linked Credential records via `credential.to_esperanto_config()`.

### Can I switch between local models (Ollama) and cloud providers (OpenAI) without code changes?

Yes. Since model definitions reside in SurrealDB and the `ModelManager` normalizes provider names (converting underscores to hyphens), you can configure an Ollama model in the database with `provider="ollama"` and the system will automatically set `OLLAMA_API_BASE` via the key provider. The `AIFactory.create_language()` method instantiates the appropriate client regardless of whether the target is local or cloud-based.

### What happens if a model request exceeds the context window limit?

When `provision_langchain_model` detects content exceeding **105,000 tokens**, it automatically routes the request to the configured large-context default model instead of the standard default. This check occurs before model instantiation, ensuring high-volume content always processes through an appropriate context window without manual intervention.

### Where are the default models configured in the system?

Default models are defined in the `DefaultModels` class within [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py), which stores references to specific model IDs for different operation types (chat, embeddings, transformations). The singleton `model_manager` uses these references when `provision_langchain_model` is called without an explicit `model_id` parameter, falling back to the appropriate default based on the `default_type` argument.