How to Implement Unified AI Model Provisioning with Esperanto in Open Notebook

Open Notebook implements unified AI model provisioning with Esperanto through a three-layer architecture that abstracts provider-specific details into a single, database-driven configuration system.

The lfnovo/open-notebook repository leverages the Esperanto library to normalize interactions across LLMs, embedding models, and speech services. By decoupling credential management from model instantiation, the system allows developers to switch providers or rotate API keys without modifying application code.

The Three-Layer Architecture for Unified Provisioning

Open Notebook achieves provider agnosticism through three coordinated components that handle credentials, environment setup, and model creation.

Credential Domain Model

The Credential class in open_notebook/domain/credential.py serves as the single source of truth for provider authentication. It stores per-provider settings—including API keys, base URLs, and context window sizes—in SurrealDB. The to_esperanto_config() method (lines 10-38) transforms these database records into plain dictionaries that Esperanto’s factories consume directly.

Key Provider Environment Bridge

The key_provider module in open_notebook/ai/key_provider.py bridges the gap between database-stored credentials and libraries that rely on environment variables. The provision_provider_keys() function (lines 46-81) queries the database for the first matching credential, then populates environment variables according to a static PROVIDER_CONFIG mapping (lines 28-73). This ensures backward compatibility with SDKs that read os.getenv for values like OPENAI_API_KEY or AZURE_OPENAI_ENDPOINT.

Model Manager Facade

ModelManager in open_notebook/ai/models.py provides the high-level interface that the rest of the codebase interacts with. The get_model() method (lines 2-64) resolves model IDs, merges database configuration with runtime overrides, and delegates to Esperanto’s AIFactory. This class also exposes get_default_model() (lines 221-264) for retrieving system-default models by type.

How the Unified Provisioning Flow Works

The provisioning pipeline executes in four distinct phases, ensuring that credentials are correctly mapped regardless of the underlying provider.

Step 1: Credential Serialization to Esperanto Config

When a credential record is linked to a model, Credential.to_esperanto_config() extracts fields like api_key, base_url, and num_ctx into a configuration dictionary. This dict bypasses Esperanto’s default environment variable lookup, allowing explicit control over authentication parameters.

Step 2: Environment Variable Provisioning

For models without linked credentials or for third-party compatibility, the system calls provision_provider_keys(provider). This function handles complex provider setups—including Vertex AI project IDs and Azure-specific headers—by writing the appropriate values to os.environ before model instantiation occurs.

Step 3: Dynamic Model Instantiation

The ModelManager.get_model() method orchestrates creation by:

1. Fetching the Model record from SurrealDB
2. Validating the model type (language, embedding, speech-to-text, or text-to-speech)
3. Merging database config with runtime kwargs
4. Normalizing provider names (converting underscores to hyphens)
5. Calling the appropriate Esperanto factory:

return AIFactory.create_language(
    model_name=model.name,
    provider=provider,
    config=config,
)

Similar branches exist for create_embedding(), create_speech_to_text(), and create_text_to_speech().

Step 4: Default Model Resolution

The get_default_model() method queries the DefaultModels singleton to determine which model ID corresponds to the requested type (e.g., "chat" or "embedding"). It then delegates to get_model(), ensuring that UI components and background jobs always use the current system defaults without hardcoding provider names.

Implementation Examples

Retrieving the Default Chat Model

Use ModelManager to fetch the configured default language model with runtime parameter overrides:

from open_notebook.ai.models import model_manager

async def get_chat_llm():
    llm = await model_manager.get_default_model("chat", temperature=0.7)
    # `llm` is an instantiated Esperanto LanguageModel ready for inference

    return llm

Manual Provider Provisioning

Explicitly provision environment variables before creating a specific model by ID:

from open_notebook.ai.key_provider import provision_provider_keys
from open_notebook.ai.models import model_manager

async def use_openai_model():
    # Ensure any stored OpenAI credential is reflected in env vars

    await provision_provider_keys("openai")
    # Create a language model by its DB ID

    model = await model_manager.get_model("model:open_notebook:my-gpt4")
    # `model` is an Esperanto LanguageModel instance

    response = await model.chat("Explain unified provisioning")
    return response

Overriding Configuration at Runtime

Override database-stored parameters when loading a specific model:

async def custom_embedding():
    # Load a specific model and override its `max_input_tokens`

    embed = await model_manager.get_model(
        "model:open_notebook:my-embed-model",
        max_input_tokens=8192,
    )
    vectors = await embed.embed(["Open Notebook"], batch=True)
    return vectors

Extending the System with New Providers

Adding support for a new AI provider requires minimal changes to the existing unified provisioning infrastructure:

1. Update PROVIDER_CONFIG in open_notebook/ai/key_provider.py to map the new provider to its required environment variable names
2. Create a provisioning helper (if needed) for complex authentication flows like OAuth or regional endpoints
3. Store credentials using the Credential domain model, which automatically makes them available via to_esperanto_config()

The ModelManager, HTTP routers in api/routers/models.py, and background workers remain unchanged because they interact only with the abstracted ModelManager interface.

Summary

• Unified provisioning in Open Notebook relies on three core files: open_notebook/domain/credential.py for data storage, open_notebook/ai/key_provider.py for environment setup, and open_notebook/ai/models.py for model instantiation.
• Esperanto integration occurs through the AIFactory class, which receives normalized configuration dictionaries from the Credential model.
• Backward compatibility is maintained via provision_provider_keys(), which syncs database credentials to environment variables for legacy SDK support.
• Provider-agnostic code is achieved by resolving all model references through ModelManager rather than instantiating provider-specific clients directly.

Frequently Asked Questions

What is Esperanto in the context of Open Notebook?

Esperanto is a Python library that abstracts multiple AI providers (OpenAI, Anthropic, Azure, etc.) behind a unified interface. Open Notebook uses Esperanto's AIFactory to create standardized LanguageModel, EmbeddingModel, and speech model instances, allowing the application to switch between GPT-4, Claude, or local Ollama models without code changes.

How does ModelManager handle missing credentials?

When ModelManager.get_model() encounters a model without a linked credential, it automatically calls provision_provider_keys() to populate environment variables from the database. If no credentials exist in the database, the system falls back to manually set environment variables, preserving compatibility with external secret management systems.

Where are AI provider credentials stored?

Credentials reside in SurrealDB as Credential records, accessed through the domain model in open_notebook/domain/credential.py. This centralization ensures that API keys, base URLs, and model-specific parameters like num_ctx are managed through the database rather than scattered across configuration files.

Can I override model parameters at runtime?

Yes, the get_model() method accepts arbitrary **kwargs that merge with database-provided configuration. Parameters passed at runtime—such as temperature, max_tokens, or max_input_tokens—take precedence over stored values, enabling dynamic adjustments for specific inference tasks without modifying persistent records.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}