# How to Implement Unified AI Model Provisioning with Esperanto in Open Notebook

> Implement unified AI model provisioning with Esperanto in Open Notebook. Learn how this three-layer architecture simplifies configuration and abstracts provider details for efficient AI deployment.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-07-02

---

**Open Notebook implements unified AI model provisioning with Esperanto through a three-layer architecture that abstracts provider-specific details into a single, database-driven configuration system.**

The **lfnovo/open-notebook** repository leverages the Esperanto library to normalize interactions across LLMs, embedding models, and speech services. By decoupling credential management from model instantiation, the system allows developers to switch providers or rotate API keys without modifying application code.

## The Three-Layer Architecture for Unified Provisioning

Open Notebook achieves provider agnosticism through three coordinated components that handle credentials, environment setup, and model creation.

### Credential Domain Model

The `Credential` class in [`open_notebook/domain/credential.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py) serves as the single source of truth for provider authentication. It stores per-provider settings—including API keys, base URLs, and context window sizes—in SurrealDB. The `to_esperanto_config()` method (lines 10-38) transforms these database records into plain dictionaries that Esperanto’s factories consume directly.

### Key Provider Environment Bridge

The `key_provider` module in [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) bridges the gap between database-stored credentials and libraries that rely on environment variables. The `provision_provider_keys()` function (lines 46-81) queries the database for the first matching credential, then populates environment variables according to a static `PROVIDER_CONFIG` mapping (lines 28-73). This ensures backward compatibility with SDKs that read `os.getenv` for values like `OPENAI_API_KEY` or `AZURE_OPENAI_ENDPOINT`.

### Model Manager Facade

`ModelManager` in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py) provides the high-level interface that the rest of the codebase interacts with. The `get_model()` method (lines 2-64) resolves model IDs, merges database configuration with runtime overrides, and delegates to Esperanto’s `AIFactory`. This class also exposes `get_default_model()` (lines 221-264) for retrieving system-default models by type.

## How the Unified Provisioning Flow Works

The provisioning pipeline executes in four distinct phases, ensuring that credentials are correctly mapped regardless of the underlying provider.

### Step 1: Credential Serialization to Esperanto Config

When a credential record is linked to a model, `Credential.to_esperanto_config()` extracts fields like `api_key`, `base_url`, and `num_ctx` into a configuration dictionary. This dict bypasses Esperanto’s default environment variable lookup, allowing explicit control over authentication parameters.

### Step 2: Environment Variable Provisioning

For models without linked credentials or for third-party compatibility, the system calls `provision_provider_keys(provider)`. This function handles complex provider setups—including Vertex AI project IDs and Azure-specific headers—by writing the appropriate values to `os.environ` before model instantiation occurs.

### Step 3: Dynamic Model Instantiation

The `ModelManager.get_model()` method orchestrates creation by:

1. Fetching the `Model` record from SurrealDB
2. Validating the model type (language, embedding, speech-to-text, or text-to-speech)
3. Merging database config with runtime `kwargs`
4. Normalizing provider names (converting underscores to hyphens)
5. Calling the appropriate Esperanto factory:

```python
return AIFactory.create_language(
    model_name=model.name,
    provider=provider,
    config=config,
)

```

Similar branches exist for `create_embedding()`, `create_speech_to_text()`, and `create_text_to_speech()`.

### Step 4: Default Model Resolution

The `get_default_model()` method queries the `DefaultModels` singleton to determine which model ID corresponds to the requested type (e.g., "chat" or "embedding"). It then delegates to `get_model()`, ensuring that UI components and background jobs always use the current system defaults without hardcoding provider names.

## Implementation Examples

### Retrieving the Default Chat Model

Use `ModelManager` to fetch the configured default language model with runtime parameter overrides:

```python
from open_notebook.ai.models import model_manager

async def get_chat_llm():
    llm = await model_manager.get_default_model("chat", temperature=0.7)
    # `llm` is an instantiated Esperanto LanguageModel ready for inference

    return llm

```

### Manual Provider Provisioning

Explicitly provision environment variables before creating a specific model by ID:

```python
from open_notebook.ai.key_provider import provision_provider_keys
from open_notebook.ai.models import model_manager

async def use_openai_model():
    # Ensure any stored OpenAI credential is reflected in env vars

    await provision_provider_keys("openai")
    # Create a language model by its DB ID

    model = await model_manager.get_model("model:open_notebook:my-gpt4")
    # `model` is an Esperanto LanguageModel instance

    response = await model.chat("Explain unified provisioning")
    return response

```

### Overriding Configuration at Runtime

Override database-stored parameters when loading a specific model:

```python
async def custom_embedding():
    # Load a specific model and override its `max_input_tokens`

    embed = await model_manager.get_model(
        "model:open_notebook:my-embed-model",
        max_input_tokens=8192,
    )
    vectors = await embed.embed(["Open Notebook"], batch=True)
    return vectors

```

## Extending the System with New Providers

Adding support for a new AI provider requires minimal changes to the existing unified provisioning infrastructure:

1. **Update `PROVIDER_CONFIG`** in [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) to map the new provider to its required environment variable names
2. **Create a provisioning helper** (if needed) for complex authentication flows like OAuth or regional endpoints
3. **Store credentials** using the `Credential` domain model, which automatically makes them available via `to_esperanto_config()`

The `ModelManager`, HTTP routers in [`api/routers/models.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/models.py), and background workers remain unchanged because they interact only with the abstracted `ModelManager` interface.

## Summary

- **Unified provisioning** in Open Notebook relies on three core files: [`open_notebook/domain/credential.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py) for data storage, [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) for environment setup, and [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py) for model instantiation.
- **Esperanto integration** occurs through the `AIFactory` class, which receives normalized configuration dictionaries from the `Credential` model.
- **Backward compatibility** is maintained via `provision_provider_keys()`, which syncs database credentials to environment variables for legacy SDK support.
- **Provider-agnostic code** is achieved by resolving all model references through `ModelManager` rather than instantiating provider-specific clients directly.

## Frequently Asked Questions

### What is Esperanto in the context of Open Notebook?

**Esperanto is a Python library that abstracts multiple AI providers (OpenAI, Anthropic, Azure, etc.) behind a unified interface.** Open Notebook uses Esperanto's `AIFactory` to create standardized `LanguageModel`, `EmbeddingModel`, and speech model instances, allowing the application to switch between GPT-4, Claude, or local Ollama models without code changes.

### How does ModelManager handle missing credentials?

**When `ModelManager.get_model()` encounters a model without a linked credential, it automatically calls `provision_provider_keys()` to populate environment variables from the database.** If no credentials exist in the database, the system falls back to manually set environment variables, preserving compatibility with external secret management systems.

### Where are AI provider credentials stored?

**Credentials reside in SurrealDB as `Credential` records, accessed through the domain model in [`open_notebook/domain/credential.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py).** This centralization ensures that API keys, base URLs, and model-specific parameters like `num_ctx` are managed through the database rather than scattered across configuration files.

### Can I override model parameters at runtime?

**Yes, the `get_model()` method accepts arbitrary `**kwargs` that merge with database-provided configuration.** Parameters passed at runtime—such as `temperature`, `max_tokens`, or `max_input_tokens`—take precedence over stored values, enabling dynamic adjustments for specific inference tasks without modifying persistent records.