How to Add a New AI Provider to the Esperanto Integration in Open Notebook

Adding a new AI provider to the Esperanto integration requires registering environment variables in key_provider.py, implementing a discovery coroutine in model_discovery.py, and optionally extending the credential schema for complex authentication flows.

Open Notebook uses the Esperanto library as a unified front-end for all AI providers, abstracting provider-specific implementations behind a common interface. To integrate a new service—whether it is a proprietary LLM API, a specialized embedding provider, or a speech-to-text platform—you must wire it into three core integration points that handle credential provisioning, model discovery, and type classification. The following guide specifies the exact files and functions to modify in the lfnovo/open-notebook repository.

Register the Provider in key_provider.py

Begin by mapping your provider name to the environment variables that Esperanto expects at runtime. This mapping enables the system to inject database-stored credentials into the process environment.

In open_notebook/ai/key_provider.py, add your provider to the PROVIDER_CONFIG dictionary:

# open_notebook/ai/key_provider.py

PROVIDER_CONFIG = {
    # Existing entries ...

    "myprovider": {
        "env_var": "MYPROVIDER_API_KEY",      # simple API-key case

        # For URL-based providers add a second entry:

        # "env_var": "MYPROVIDER_API_BASE",

    },
}

When provision_provider_keys() executes, it reads this configuration to copy credentials from Credential records into environment variables. This ensures that when Esperanto initializes a model for myprovider, the required authentication tokens are available via os.getenv().

Implement Model Discovery in model_discovery.py

Next, implement an asynchronous discovery function that fetches available models from your provider's API. This automates population of the model database, making new models selectable in the UI without manual data entry.

Create a discovery coroutine in open_notebook/ai/model_discovery.py:

# open_notebook/ai/model_discovery.py

import os
import httpx
from typing import List

async def discover_myprovider_models() -> List[DiscoveredModel]:
    """Fetch the list of models from MyProvider's /v1/models endpoint."""
    api_key = os.getenv("MYPROVIDER_API_KEY")
    base_url = os.getenv("MYPROVIDER_API_BASE", "https://api.myprovider.com")
    
    if not api_key:
        return []  # no credentials → no discovery

    models: List[DiscoveredModel] = []
    try:
        async with httpx.AsyncClient() as client:
            resp = await client.get(
                f"{base_url}/v1/models",
                headers={"Authorization": f"Bearer {api_key}"},
                timeout=30.0,
            )
            resp.raise_for_status()
            for m in resp.json().get("models", []):
                name = m.get("id")
                if name:
                    model_type = classify_model_type(name, "myprovider")
                    models.append(
                        DiscoveredModel(
                            name=name,
                            provider="myprovider",
                            model_type=model_type,
                        )
                    )
    except Exception as exc:
        logger.warning(f"Failed to discover MyProvider models: {exc}")
    return models

After defining the function, register it in the PROVIDER_DISCOVERY_FUNCTIONS dictionary at the bottom of the same file:

# open_notebook/ai/model_discovery.py

PROVIDER_DISCOVERY_FUNCTIONS["myprovider"] = discover_myprovider_models

The sync_all_providers() function iterates over this registry to automatically populate the model table in SurrealDB when users trigger the "Sync Models" action in the UI.

Classify Model Types

To ensure Open Notebook correctly identifies whether a model handles language, embeddings, speech-to-text (STT), or text-to-speech (TTS), extend the classify_model_type function with provider-specific naming patterns.

Add your provider's conventions to the type mappings in open_notebook/ai/model_discovery.py:

# Inside classify_model_type function

MYPROVIDER_MODEL_TYPES = {
    "language": ["gpt", "chat", "llm"],
    "embedding": ["embed"],
    "speech_to_text": ["stt"],
    "text_to_speech": ["tts"],
}

# Add to the type_mappings dict

type_mappings["myprovider"] = MYPROVIDER_MODEL_TYPES

Accurate classification ensures that ModelManager.get_default_model() returns the correct model type—such as LanguageModel or EmbeddingModel—when workflows request specific capabilities.

Handle Complex Credentials (Optional)

If your provider requires additional parameters beyond a simple API key—such as project IDs, regions, or service account JSON—you must extend the Credential domain model.

In open_notebook/domain/credential.py, add the necessary fields and update the to_esperanto_config method:

# open_notebook/domain/credential.py

from typing import Optional, Dict, Any

class Credential(ObjectModel):
    # ... existing fields ...

    myprovider_project: Optional[str] = None   # new field

    
    def to_esperanto_config(self) -> Dict[str, Any]:
        cfg = super().to_esperanto_config()
        if self.myprovider_project:
            cfg["project"] = self.myprovider_project
        return cfg

The ModelManager calls to_esperanto_config() when instantiating models, passing the resulting dictionary to Esperanto's factory methods. This allows complex authentication flows to work seamlessly with the unified interface.

End-to-End Workflow

Once the integration points are implemented, the execution flow proceeds as follows:

1. Credential Provisioning: When a model is requested, provision_provider_keys() sets environment variables from Credential records using the PROVIDER_CONFIG mapping.
2. Model Discovery: The sync_provider_models("myprovider") coroutine calls your discovery function and inserts Model records into SurrealDB via Model.save().
3. UI Registration: The frontend automatically displays discovered models under Settings → Models because the UI pulls from /api/routers/models which uses ModelManager.
4. Runtime Instantiation: When workflows need a model, ModelManager.get_model() loads the record, provisions credentials, and calls AIFactory.create_*() with the provider name and configuration.

Summary

• Register environment variables in open_notebook/ai/key_provider.py by adding an entry to PROVIDER_CONFIG that maps your provider name to its required environment variables.
• Implement discovery in open_notebook/ai/model_discovery.py by writing an async function that fetches model IDs from your provider's API and registering it in PROVIDER_DISCOVERY_FUNCTIONS.
• Classify model types by extending classify_model_type with provider-specific naming patterns to ensure correct model instantiation.
• Extend credentials in open_notebook/domain/credential.py if your provider requires complex authentication parameters beyond a simple API key.
• No UI changes required: The frontend automatically surfaces new models once they are discovered and saved to the database.

Frequently Asked Questions

Do I need to modify the frontend code to add a new AI provider?

No. The React frontend automatically displays any models discovered through the API endpoints. Once you implement the discovery function in model_discovery.py and run the sync process, models appear in the Settings → Models interface without requiring UI modifications because the system pulls data from ModelManager.

What if my provider doesn't have a model listing endpoint?

If your provider lacks a discovery API, modify your discovery function to return a static list of known models. Return List[DiscoveredModel] with hardcoded entries for the models you want to support, and the system will treat them exactly like dynamically discovered models, inserting them into SurrealDB via Model.save().

How does the system handle different model types like embeddings versus language models?

The classify_model_type function in model_discovery.py analyzes model names using substring patterns specific to each provider. When you add your provider to the type_mappings dictionary, the system categorizes models correctly, ensuring that ModelManager instantiates the appropriate Esperanto class (e.g., EmbeddingModel vs LanguageModel) based on the classified type.

Can I add a provider that requires OAuth or service account authentication instead of API keys?

Yes. Extend the Credential class in open_notebook/domain/credential.py with fields for tokens, refresh tokens, or file paths. Override to_esperanto_config() to transform these fields into the configuration dictionary format that Esperanto expects. The credential system supports arbitrary key-value pairs, allowing complex authentication flows while maintaining the same unified interface.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}