# Multi-Provider AI Provisioning Architecture: How Open Notebook Uses Esperanto and ModelManager

> Discover Open Notebooks multi-provider AI provisioning architecture. Learn how Esperanto and ModelManager dynamically instantiate LangChain models from eight providers in this technical guide.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: architecture
- Published: 2026-06-21

---

**Open Notebook provisions AI models through a four-layer pipeline that combines request-time token analysis, database-driven defaults, secure credential injection, and the Esperanto unified API to dynamically instantiate LangChain-compatible models from over eight providers.**

Open Notebook is an open-source knowledge management system that abstracts away provider-specific complexities when working with language models. Its **multi-provider AI provisioning architecture** seamlessly routes requests through the Esperanto library and `ModelManager` class, enabling dynamic model selection based on content size while maintaining secure credential storage in SurrealDB.

## The Four-Layer Provisioning Pipeline

The architecture follows a linear delegation chain from request to runtime model:

- **Request Analysis** – Token counting and model selection logic
- **Model Resolution** – Database lookups via `ModelManager`
- **Credential Injection** – Environment variable population from encrypted storage
- **Factory Instantiation** – Esperanto object creation and LangChain conversion

## Entry Point: Content-Aware Model Selection

The provisioning process begins in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) with the `provision_langchain_model` function. This helper determines which model identifier to request based on runtime conditions.

When invoked, the function first analyzes the token length of the supplied content. If the count exceeds **105,000 tokens**, the system automatically selects a **large-context** default model rather than the standard chat model. This prevents context window overflows during document processing workflows.

If the caller explicitly provides a `model_id` parameter, that specific model takes precedence. Otherwise, the function falls back to the default model for the requested `default_type` (e.g., `"chat"`, `"embedding"`, `"text_to_speech"`, or `"transformation"`).

## The ModelManager Layer: Database-to-Runtime Bridge

Once a model identifier is selected, control passes to the `ModelManager` class in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py). This manager transforms database records into concrete Esperanto model objects capable of inference.

### Default Model Resolution

The `get_default_model` method queries the `DefaultModels` singleton stored in SurrealDB. This singleton maintains the current default model ID for each model type, allowing administrators to switch provider configurations without code changes.

The `get_model` method then loads the specific `Model` record, which contains fields for `name`, `provider`, `type`, and an optional `credential` reference.

### Secure Credential Injection

When a model record links to a credential, `ModelManager` delegates to `provision_provider_keys` in [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py). This function maps the provider type to its required environment variables:

- **Simple providers** (OpenAI, Anthropic): Sets `OPENAI_API_KEY` or `ANTHROPIC_API_KEY`
- **Complex providers** (Azure, Vertex): Injects endpoint URLs, project IDs, and auxiliary configuration variables alongside API keys

If no credential exists in the database, the system gracefully falls back to pre-existing environment variables, preserving backward compatibility with shell-based configuration.

## Esperanto Integration: Provider-Agnostic Model Factory

After environment preparation, `ModelManager` invokes the appropriate `AIFactory` method based on the model's `type` field:

- `AIFactory.create_language()` for chat and completion models
- `AIFactory.create_embedding()` for vector generation
- `AIFactory.create_speech_to_text()` and `AIFactory.create_text_to_speech()` for audio processing

This factory pattern allows Open Notebook to support **over eight AI providers** (including OpenAI, Anthropic, Google, and Azure) through a single code path, abstracting provider-specific implementation details behind the Esperanto unified interface.

## LangChain Integration and Workflow Consumption

The final step in `provision_langchain_model` calls `model.to_langchain()` on the returned `LanguageModel` wrapper. This method exposes a LangChain `BaseChatModel` interface that integrates directly with downstream LangGraph workflows for source ingestion, chat completions, and document transformations.

The complete provisioning chain follows this execution flow:

```

request → provision_langchain_model → ModelManager.get_default_model / get_model
      → (optional) provision_provider_keys → AIFactory.create_* → LanguageModel
      → to_langchain() → LangChain model

```

## Practical Implementation Examples

The following examples demonstrate common provisioning patterns in the Open Notebook codebase:

```python

# Automatic large-context detection for long documents

await provision_langchain_model(
    content=very_long_text,
    model_id=None,
    default_type="chat",
)

```

```python

# Explicit model selection by database ID

await provision_langchain_model(
    content=short_prompt,
    model_id="model:openai:gpt-4o",
    default_type="chat",
)

```

```python

# Direct embedding model retrieval via ModelManager

from open_notebook.ai.models import model_manager

embedding_model = await model_manager.get_embedding_model()

# Returns an Esperanto EmbeddingModel ready for vector generation

```

```python

# Manual credential provisioning for custom workflows

await provision_provider_keys("anthropic")

# Subsequent AIFactory calls automatically pick up ANTHROPIC_API_KEY

```

## Key Source Files

Understanding the architecture requires familiarity with these specific modules:

- **[`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py)** – Entry point for model selection and LangChain conversion
- **[`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py)** – `ModelManager` class, `DefaultModels` singleton, and database schema definitions
- **[`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py)** – Credential-to-environment mapping for all supported providers
- **[`open_notebook/domain/credential.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py)** – Domain model for secure API key storage

## Summary

- **Dynamic Content Routing**: The `provision_langchain_model` function automatically selects large-context models when content exceeds 105,000 tokens.
- **Database-Driven Configuration**: The `ModelManager` class reads default models from SurrealDB's `DefaultModels` singleton, enabling runtime provider switching without deployment changes.
- **Secure Credential Management**: The `key_provider` module injects provider-specific environment variables from encrypted `Credential` records while maintaining fallback support for shell environment variables.
- **Unified Provider Interface**: Esperanto's `AIFactory` methods abstract over eight providers into a consistent API for language, embedding, and speech models.
- **Seamless LangChain Integration**: All provisioned models expose a `to_langchain()` method for immediate use in LangGraph workflows.

## Frequently Asked Questions

### How does Open Notebook handle API key rotation without restarting services?

The `ModelManager` calls `provision_provider_keys` during every model provisioning request, reading the latest credential values from SurrealDB. Because environment variables are set at request time rather than startup, updated API keys in the `Credential` table take effect immediately for subsequent model instantiations.

### What happens if a requested model ID does not exist in the database?

The `get_model` method in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py) loads the specific record by ID. If the record does not exist or is invalid, the database query will fail, and the provisioning chain will raise an appropriate error before attempting to construct the Esperanto object, ensuring that only configured models are instantiated.

### Can Open Notebook mix providers within a single workflow?

Yes. Since `provision_langchain_model` and `ModelManager` are stateless with respect to providers, different steps in a LangGraph workflow can call the provisioning functions with different `model_id` values or `default_type` parameters. Each call independently resolves credentials and configures the appropriate environment variables for its specific provider.

### Why use Esperanto instead of calling provider SDKs directly?

Esperanto provides a unified factory interface (`AIFactory`) that normalizes the instantiation patterns across providers like OpenAI, Anthropic, Google, and Azure. This allows Open Notebook to support new providers by adding database records and credential mappings rather than modifying workflow code, significantly reducing the maintenance burden for multi-provider AI infrastructure.