# Implementing Model Fallback Logic When the Primary AI Provider Fails

> Implement AI model fallback logic automatically with Open-Notebook. Seamlessly provision API keys from SurrealDB or env vars when your primary AI provider fails, ensuring graceful degradation without code changes.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-06-22

---

**Open-Notebook implements a credential-first, environment-variable fallback system that automatically provisions API keys from SurrealDB or process environment variables when the primary AI provider fails, allowing graceful degradation without code changes.**

Open-Notebook abstracts every AI service—LLM, embedding, speech-to-text, and text-to-speech—behind a *model* record stored in SurrealDB. When a model is requested, the `ModelManager` performs a multi-step resolution process that prioritizes stored credentials and falls back to environment variables, ensuring continuous operation even when primary providers experience outages.

## How the ModelManager Resolves AI Provider Credentials

The `ModelManager` class orchestrates the fallback flow through three distinct resolution steps. This architecture ensures that secrets remain out of the repository while maintaining operational flexibility.

### Step 1: Loading the Model Definition

When a model is requested, the system first retrieves its configuration. In [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py), the `Model.get(model_id)` method fetches the record from SurrealDB, returning the model's name, provider, type, and optional credential linkage.

### Step 2: Credential Resolution from SurrealDB

If the model record includes a linked credential, the manager loads the `Credential` object via `model.get_credential_obj()`. This **credential-first** approach keeps sensitive API keys securely stored in the database rather than in environment variables or code.

### Step 3: Environment Variable Fallback

When the credential cannot be loaded or when no credential is attached, the manager invokes `provision_provider_keys(provider)`. Defined in [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) (lines 254-260), this helper first searches for stored API keys in the database; if none are found, it pulls the required variables from the process environment—such as `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, or `COHERE_API_KEY`.

## The Fallback Implementation in Key Files

The fallback logic is explicitly codified in two critical modules that handle the transition from database credentials to environment variables.

### Credential-to-Env Resolution in models.py

In [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py) (lines 122-138), the `ModelManager` decides whether to use a stored credential or fall back to environment variables. This logic creates the final `config` dictionary that combines credential data with any caller-supplied kwargs before invoking the appropriate `AIFactory.create_*` method.

### Database-First Provisioning in key_provider.py

The `provision_provider_keys` function in [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) implements the DB-first, env-var fallback pattern. This ensures that development or CI environments can run without persisting credentials to the database, while production deployments can leverage secure credential storage.

The API router for the Models UI mirrors this same pattern in [`api/routers/models.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/models.py) (line 370), checking database credentials first and then environment variables when serving the UI.

## Practical Code Examples

The following examples demonstrate how to leverage the fallback mechanism in application code and API endpoints.

### Retrieving the Default Chat Model with Fallback

This example shows how to request the default chat model, which automatically triggers the credential-to-environment-variable fallback chain if the stored credential is unavailable:

```python
from open_notebook.ai.models import model_manager

async def get_chat_model():
    # Will use the model ID stored in DefaultModels.default_chat_model

    chat_model = await model_manager.get_default_model("chat")
    # If the linked credential is missing, provision_provider_keys()

    # loads OPENAI_API_KEY (or another provider key) from the environment.

    return chat_model

```

### Requesting a Specific Model with Environment Fallback

When requesting a specific embedding model that may point to a non-existent credential, the system automatically invokes the fallback logic:

```python
from open_notebook.ai.models import model_manager

async def request_embedding():
    # Assume we have a model record that points to a non‑existent credential

    embedding = await model_manager.get_model("model:my_embedding")
    # The call above will automatically invoke provision_provider_keys()

    # which reads OPENAI_API_KEY, COHERE_API_KEY, etc. from the env.

    return embedding

```

### Handling Configuration Errors in API Endpoints

When exposing models through the REST API, catch `ConfigurationError` to provide clear feedback when neither database credentials nor environment variables are available:

```python

# Example: API endpoint that returns a model or a clear error

# (excerpt from api/routers/models.py)

from open_notebook.ai.models import model_manager
from starlette.exceptions import HTTPException

@router.get("/{model_id}")
async def read_model(model_id: str):
    try:
        model = await model_manager.get_model(model_id)
        return model
    except ConfigurationError as exc:
        raise HTTPException(status_code=400, detail=str(exc))

```

## Architectural Benefits of the Fallback Pattern

The Open-Notebook architecture ensures **graceful degradation** without requiring code changes. If a provider's service is down, the application can be re-configured to point to an alternate provider simply by updating the model record or environment variables.

This design provides three key advantages:

1. **Credential-first resolution** keeps secrets out of the repository and version control
2. **Env-var fallback** guarantees that development or CI environments can operate without persisting credentials to the database
3. **Runtime flexibility** allows operators to switch providers instantly by changing environment variables or updating the `DefaultModels` configuration

## Summary

Open-Notebook's model fallback mechanism provides a robust, production-ready solution for handling AI provider failures:

- **Credential-first resolution** via `Model.get()` and `get_credential_obj()` prioritizes secure database storage
- **Automatic env-var fallback** through `provision_provider_keys()` in [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) ensures continuous operation
- **Zero-code reconfiguration** allows switching providers by updating model records or environment variables
- **Consistent API behavior** across `model_manager.get_model()` and `model_manager.get_default_model()` methods

## Frequently Asked Questions

### How does Open-Notebook handle missing credentials when the primary AI provider fails?

When the primary provider fails or credentials are missing, the `ModelManager` automatically invokes `provision_provider_keys()` to retrieve API keys from environment variables such as `OPENAI_API_KEY` or `ANTHROPIC_API_KEY`. This fallback occurs in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py) (lines 122-138) without requiring changes to application code.

### What environment variables are supported for the fallback mechanism?

The system supports provider-specific environment variables including `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, and `COHERE_API_KEY`. The `provision_provider_keys()` function in [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) (lines 254-260) dynamically retrieves these based on the provider specified in the model configuration.

### Can I use multiple fallback providers for the same model type?

While the architecture supports reconfiguring the fallback provider by updating the model record in SurrealDB or changing environment variables, the current implementation selects one provider per model record. To switch providers, update the `DefaultModels` configuration or the specific model ID in your code to point to an alternative provider's model record.

### Where is the fallback logic implemented in the Open-Notebook codebase?

The core fallback logic resides in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py) for the credential resolution decision (lines 122-138) and [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py) for the database-to-environment provisioning (lines 254-260). The API layer in [`api/routers/models.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/models.py) (line 370) mirrors this pattern for UI-facing endpoints.