How to Add a New AI Provider to the Esperanto Integration in Open Notebook

To add a new AI provider to Open Notebook's Esperanto integration, register the provider's environment variables in key_provider.py, implement a discovery coroutine in model_discovery.py, and optionally extend the Credential class for complex authentication flows.

Open Notebook leverages the Esperanto library as a unified front-end for all AI providers, handling language models, embeddings, and speech services through a single abstraction layer. When you add a new AI provider to the Esperanto integration, the system automatically discovers available models, classifies them by capability, and exposes them through the existing UI without requiring front-end changes. This guide walks through the exact files and functions you need to modify in the lfnovo/open-notebook repository.

Register the Provider Configuration

Start by mapping your provider's authentication requirements in open_notebook/ai/key_provider.py. The PROVIDER_CONFIG dictionary tells the system which environment variables Esperanto expects for each provider.

Add an entry that maps the provider name to its environment variable(s):

# open_notebook/ai/key_provider.py

PROVIDER_CONFIG = {
    # Existing entries...

    "myprovider": {
        "env_var": "MYPROVIDER_API_KEY",
        # For URL-based providers, add additional entries:

        # "base_url_var": "MYPROVIDER_API_BASE",

    },
}

When provision_provider_keys() runs, it looks up PROVIDER_CONFIG to copy database-stored credentials into the process environment. This ensures that when Esperanto initializes a model for myprovider, the API key is available via os.getenv("MYPROVIDER_API_KEY").

Implement Model Discovery

Model discovery automatically populates the database with available models from your provider, making them selectable in the UI without manual data entry.

Create the Discovery Coroutine

In open_notebook/ai/model_discovery.py, implement an async function that fetches the model list from your provider's API. If the provider offers a /v1/models endpoint (similar to OpenAI), use this pattern:

# open_notebook/ai/model_discovery.py

import os
import httpx
from typing import List
from open_notebook.ai.models import DiscoveredModel

async def discover_myprovider_models() -> List[DiscoveredModel]:
    """Fetch available models from MyProvider's API."""
    api_key = os.getenv("MYPROVIDER_API_KEY")
    base_url = os.getenv("MYPROVIDER_API_BASE", "https://api.myprovider.com")
    
    if not api_key:
        return []
    
    models: List[DiscoveredModel] = []
    try:
        async with httpx.AsyncClient() as client:
            resp = await client.get(
                f"{base_url}/v1/models",
                headers={"Authorization": f"Bearer {api_key}"},
                timeout=30.0,
            )
            resp.raise_for_status()
            
            for m in resp.json().get("models", []):
                name = m.get("id")
                if name:
                    model_type = classify_model_type(name, "myprovider")
                    models.append(
                        DiscoveredModel(
                            name=name,
                            provider="myprovider",
                            model_type=model_type,
                        )
                    )
    except Exception as exc:
        logger.warning(f"Failed to discover MyProvider models: {exc}")
    
    return models

If your provider does not expose a model listing API, return a static list of known model IDs instead of making an HTTP request.

Register the Discovery Function

Add your coroutine to the PROVIDER_DISCOVERY_FUNCTIONS registry at the bottom of model_discovery.py:

# open_notebook/ai/model_discovery.py

PROVIDER_DISCOVERY_FUNCTIONS["myprovider"] = discover_myprovider_models

The sync_all_providers() function iterates over this dictionary to automatically populate the Model table in SurrealDB.

Classify Model Types

Teach the system how to categorize your provider's models by extending classify_model_type in model_discovery.py. This ensures that ModelManager.get_default_model() returns the correct model class (LanguageModel, EmbeddingModel, etc.).

Add a mapping for your provider's naming conventions:

# Inside open_notebook/ai/model_discovery.py

MYPROVIDER_MODEL_TYPES = {
    "language": ["gpt", "chat", "llm"],
    "embedding": ["embed", "embedding"],
    "speech_to_text": ["stt", "whisper"],
    "text_to_speech": ["tts", "voice"],
}

# Add to the type_mappings dictionary within classify_model_type()

type_mappings["myprovider"] = MYPROVIDER_MODEL_TYPES

The classifier inspects model IDs for these substrings. For example, a model named myprovider/gpt-4 would be classified as a language model, while myprovider/embed-large would be classified as an embedding model.

Handle Complex Authentication (Optional)

If your provider requires more than a simple API key (such as Azure's endpoint URLs or Vertex AI service accounts), extend the Credential domain model in open_notebook/domain/credential.py:

# open_notebook/domain/credential.py

from typing import Optional, Dict, Any

class Credential(ObjectModel):
    # Existing fields...

    myprovider_project: Optional[str] = None
    myprovider_region: Optional[str] = None
    
    def to_esperanto_config(self) -> Dict[str, Any]:
        cfg = super().to_esperanto_config()
        if self.provider == "myprovider":
            if self.myprovider_project:
                cfg["project"] = self.myprovider_project
            if self.myprovider_region:
                cfg["region"] = self.myprovider_region
        return cfg

The to_esperanto_config() method builds the configuration dictionary that Esperanto receives during model initialization. Additional fields automatically appear in the credential form UI based on the provider type.

End-to-End Integration Flow

Understanding the complete data flow helps troubleshoot integration issues:

1. Credential Storage: Users create credentials via the UI, which stores encrypted values in SurrealDB.
2. Environment Provisioning: When a model is requested, provision_provider_keys() loads the credential and sets the corresponding environment variables defined in PROVIDER_CONFIG.
3. Model Discovery: Calling sync_provider_models("myprovider") executes your discovery coroutine, inserts Model records into the database, and makes them available in Settings → Models.
4. Runtime Instantiation: When workflows request a model, ModelManager.get_model() loads the configuration, provisions credentials, and calls AIFactory.create_language_model() (or the appropriate model type) from the Esperanto library.

Summary

• Register environment variables in open_notebook/ai/key_provider.py by adding an entry to PROVIDER_CONFIG so provision_provider_keys() can inject credentials.
• Implement discovery logic in open_notebook/ai/model_discovery.py by creating an async function that returns List[DiscoveredModel] and registering it in PROVIDER_DISCOVERY_FUNCTIONS.
• Classify model capabilities by extending classify_model_type() with provider-specific naming patterns to ensure correct model type detection.
• Extend credentials in open_notebook/domain/credential.py if the provider requires additional authentication fields beyond a single API key.

Frequently Asked Questions

Do I need to modify the UI code to add a new AI provider?

No. The UI automatically populates the "Models → Add" screen by querying the /api/routers/models endpoint, which uses ModelManager to read discovered models from the database. As long as you register the discovery function in PROVIDER_DISCOVERY_FUNCTIONS, the models appear automatically. You only need to modify the UI if you want custom credential form fields, which are driven by the Credential class definition.

What if the provider doesn't have a model listing API?

Return a static list from your discovery function instead of making an HTTP request. Hardcode the known model IDs as DiscoveredModel objects with the appropriate provider name and classified types. The system will treat these as if they were fetched from an API, storing them in the database and exposing them in the model selection UI.

How are credentials securely injected into the environment?

The provision_provider_keys() function in open_notebook/ai/key_provider.py reads the Credential record from SurrealDB and temporarily sets the environment variables defined in PROVIDER_CONFIG for the current process. These values are never logged or exposed to the client; they exist only in the server-side environment where Esperanto initializes the provider client.

Can I add a provider that requires multiple authentication fields?

Yes. Extend the Credential class in open_notebook/domain/credential.py with additional fields (such as project_id, region, or organization). Override to_esperanto_config() to marshal these fields into the configuration dictionary that Esperanto expects. The discovery and provisioning logic remains the same, but the credential form will now capture the additional required information.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}