# How ModelManager Provisions AI Models with Fallback Logic in Open Notebook

> Learn how ModelManager provisions AI models with fallback logic in Open Notebook. It uses SurrealDB or environment variables for seamless model initialization.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: internals
- Published: 2026-06-18

---

**The ModelManager in the open-notebook repository provisions AI models by first attempting to load per-model credentials from SurrealDB, then automatically falling back to environment variables when credentials are unavailable or invalid, ensuring seamless initialization of language, embedding, and speech models.**

The ModelManager serves as the central bridge between database-stored model configurations and executable AI instances in the `lfnovo/open-notebook` project. This component transforms static model records into concrete Esperanto AI objects while implementing robust fallback logic to handle missing credentials gracefully. Understanding how ModelManager provisions AI models with fallback logic helps developers configure reliable authentication strategies that work across diverse deployment environments.

## Step-by-Step Model Provisioning Flow

The provisioning process implemented in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py) follows a strict sequence that prioritizes security through credential records while maintaining flexibility via environment variable fallbacks.

### Retrieve and Validate the Model Record

The process begins when `ModelManager.get_model()` receives a model identifier and loads the corresponding record from SurrealDB. The manager immediately validates that the model type belongs to the supported set of language, embedding, speech-to-text, or text-to-speech implementations.

```python
model: Model = await Model.get(model_id)
if model.type not in ["language", "embedding", "speech_to_text", "text_to_speech"]:
    raise ConfigurationError(f"Invalid model type: {model.type}")

```

This validation occurs at lines 108-118 of [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py), ensuring that only compatible model types proceed to the configuration phase.

### Attempt Credential-Based Configuration

When the model record references a stored credential, the manager attempts to load it via `model.get_credential_obj()` and convert it to an Esperanto-compatible configuration dictionary using `Credential.to_esperanto_config()`. This method, defined in [`open_notebook/domain/credential.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/credential.py), extracts API keys and provider settings from the encrypted credential store.

```python
credential = await model.get_credential_obj()
config = credential.to_esperanto_config()

```

This credential resolution occurs at lines 122-126, providing the most secure authentication path by keeping sensitive keys out of environment variables.

### Fallback to Environment Variable Provisioning

If no credential is linked to the model or if the credential load fails, the manager executes its fallback logic by calling `provision_provider_keys()` from [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py). This helper function reads the required API keys from environment variables and registers them directly with the Esperanto framework, ensuring the downstream factory call will locate the necessary secrets.

```python
from open_notebook.ai.key_provider import provision_provider_keys
await provision_provider_keys(model.provider)

```

This fallback mechanism at lines 130-138 guarantees that models remain functional even when database credentials are missing or corrupted, logging a warning to alert administrators of the automatic fallback.

### Merge Caller Configuration and Normalize Provider

After establishing the base configuration through either credentials or environment variables, the manager merges any caller-supplied parameters such as `temperature` or `max_tokens`. It then normalizes the provider name by replacing underscores with hyphens to match Esperanto's naming conventions.

```python
config.update(kwargs)
provider = model.provider.replace("_", "-")

```

These operations occur at lines 144-148, allowing runtime overrides while ensuring provider compatibility.

### Factory Instantiation with Caching

The final step dispatches to the appropriate `AIFactory` method based on the model type. Esperanto internally caches the resulting model instance, making subsequent calls performant.

```python

# Language model example from lines 151-156

return AIFactory.create_language(
    provider=provider,
    model_name=model.model_name,
    config=config,
)

```

Analogous branches exist for `create_embedding`, `create_speech_to_text`, and `create_text_to_speech` models.

## Default Model Helpers

Beyond explicit model retrieval, the ModelManager provides convenience methods that automatically resolve default configurations. The `get_default_model()`, `get_embedding_model()`, and `get_speech_to_text()` helpers read default model IDs from the `DefaultModels` record (lines 78-86) and delegate to `get_model()`.

```python
defaults = await self.get_defaults()
model_id = defaults.default_chat_model  # Lines 331-334

```

If a default ID is missing or invalid, the manager logs a warning and returns `None`, allowing the caller to implement application-specific fallback behavior.

## Practical Implementation Examples

The following examples demonstrate the provisioning flow in practice, including the automatic fallback behavior.

### Provision a Specific Language Model

```python
from open_notebook.ai.models import ModelManager

async def get_chat():
    manager = ModelManager()
    # Automatically falls back to env vars if credential is missing

    chat_model = await manager.get_model("model:12345", temperature=0.2)
    response = await chat_model.chat(messages=[...])
    return response

```

### Use the Default Embedding Model

```python
async def embed_text(text: str):
    manager = ModelManager()
    embedder = await manager.get_embedding_model()
    if embedder is None:
        raise RuntimeError("No embedding model configured")
    vector = await embedder.embed(text)
    return vector

```

### Handle Missing Credential Fallback

```python

# This triggers the fallback warning and loads from env vars

async def speech_to_text(audio_bytes):
    manager = ModelManager()
    stt = await manager.get_speech_to_text()
    transcript = await stt.transcribe(audio_bytes)
    return transcript

```

## Summary

- **ModelManager** in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py) orchestrates the transformation of database records into executable AI models.
- **Credential priority**: The system first attempts to load encrypted credentials via `Credential.to_esperanto_config()`, then falls back to `provision_provider_keys()` for environment variable authentication.
- **Provider normalization**: Underscores in provider names are converted to hyphens to satisfy Esperanto requirements before factory invocation.
- **Caching**: The `AIFactory` caches model instances internally, ensuring repeated provisioning calls remain performant.
- **Default model helpers**: Convenience methods retrieve default configurations from the `DefaultModels` record, returning `None` when defaults are undefined.

## Frequently Asked Questions

### What triggers the fallback to environment variables in ModelManager?

The fallback triggers when `model.get_credential_obj()` returns `None` or raises an exception, typically because the credential record was deleted or the reference is invalid. At this point, the manager logs a warning and executes `provision_provider_keys()` from [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) to load API keys from environment variables.

### How does ModelManager normalize provider names for Esperanto compatibility?

Database entries store provider names with underscores (e.g., `open_ai`), while Esperanto expects hyphenated formats (e.g., `open-ai`). The manager performs a string replacement at lines 147-148: `provider = model.provider.replace("_", "-")`, ensuring consistent naming before passing to `AIFactory`.

### What is the difference between credential-based and environment-variable provisioning?

Credential-based provisioning retrieves encrypted API keys from the SurrealDB database via the `Credential` domain object, offering per-model authentication suitable for multi-tenant environments. Environment-variable provisioning uses `provision_provider_keys()` to read keys from the process environment, better suited for single-tenant deployments or containerized applications.

### How are default models configured and retrieved in Open Notebook?

Default models are stored in the `DefaultModels` record, which the manager accesses via `get_defaults()` at lines 78-86. The convenience methods `get_default_model()`, `get_embedding_model()`, and similar helpers extract specific default IDs from this record (e.g., lines 331-334) and delegate to `get_model()` for actual provisioning. If no default is configured, the method returns `None` after logging a warning.