# How the Open-Notebook REST API Handles Model Configuration Overrides Per Request

> Discover how the Open-Notebook REST API manages model configuration overrides per request. Learn about its three-tier fallback system for flexible model selection.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: api-reference
- Published: 2026-06-18

---

**The Open-Notebook REST API implements a three-tier fallback system that prioritizes request-level model overrides, falls back to session-specific configurations stored in SurrealDB, and finally defaults to global settings, injecting the resolved model ID directly into the LangGraph execution context.**

The open-notebook repository provides a flexible REST API for conversational AI interactions that allows dynamic LLM selection without restarting services or modifying environment variables. By supporting per-request model configuration overrides, the system enables developers to switch between GPT-4, Claude, or other providers on a per-message basis while maintaining persistent session preferences. This architecture ensures that individual requests can specify exactly which model should handle the inference, with clear precedence rules determining the final selection.

## Where the API Accepts Model Overrides

### Chat Endpoints

In [`api/routers/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/chat.py), the Pydantic request models define an optional `model_override` field across all relevant operations. The `CreateSessionRequest`, `UpdateSessionRequest`, and `ExecuteChatRequest` schemas each accept this parameter, allowing clients to specify a model identifier such as "gpt-4o" or "claude-3-opus" at different stages of the conversation lifecycle.

### Source Chat Endpoints

The same pattern appears in [`api/routers/source_chat.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/source_chat.py), ensuring consistent behavior across different conversational interfaces. Whether initiating a new session or executing a message against an existing one, the API surface accepts the override uniformly.

## The Three-Tier Resolution Order

The resolution logic follows a strict precedence hierarchy to determine the effective model:

1. **Request-level override** – The explicit `model_override` value provided in the current HTTP request takes highest priority.
2. **Session-level override** – If the request omits the override, the system checks the `session.model_override` attribute stored in the database.
3. **Global default** – When neither request nor session specifies a model, the application falls back to the `DefaultModels` singleton accessed via `open_notebook.ai.models.DefaultModels`.

This cascade ensures maximum flexibility while maintaining sensible defaults.

## Propagation to LangGraph Execution

Once resolved, the model identifier propagates through the execution stack. In [`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py), the system injects the `model_override` into the LangGraph invocation via the `RunnableConfig` configurable dictionary:

```python
result = chat_graph.invoke(
    input=state_values,
    config=RunnableConfig(
        configurable={"thread_id": full_session_id, "model_id": model_override},
    ),
)

```

The Esperanto provider receives this `model_id` and instantiates the appropriate LLM client for that specific call, enabling true per-request model switching without thread contamination.

## Persistence and Session Management

Updates to a session's default model persist to SurrealDB. When calling `PUT /chat/sessions/{session_id}`, the `UpdateSessionRequest` payload writes the `model_override` value directly to the `ChatSession.model_override` field in [`open_notebook/domain/notebook.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/notebook.py). Subsequent requests that omit an explicit override automatically inherit this stored preference.

## Implementation Details

The resolution logic in [`api/routers/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/chat.py) implements the fallback chain explicitly:

```python
model_override = (
    request.model_override
    if request.model_override is not None
    else getattr(session, "model_override", None)
)

```

The resolved value then populates the state dictionary passed to the graph:

```python
state_values["model_override"] = model_override

```

### Practical API Examples

Create a session with a specific default model:

```bash
curl -X POST http://localhost:5055/chat/sessions \
  -H "Content-Type: application/json" \
  -d '{
        "notebook_id": "my-notebook",
        "title": "Demo Session",
        "model_override": "gpt-4o"
      }'

```

Execute a message overriding the session default:

```bash
curl -X POST http://localhost:5055/chat/execute \
  -H "Content-Type: application/json" \
  -d '{
        "session_id": "12345",
        "message": "Explain quantum computing",
        "model_override": "claude-3-opus"
      }'

```

Update an existing session's default:

```bash
curl -X PUT http://localhost:5055/chat/sessions/12345 \
  -H "Content-Type: application/json" \
  -d '{
        "model_override": "gpt-4"
      }'

```

Rely on automatic fallback:

```bash
curl -X POST http://localhost:5055/chat/execute \
  -H "Content-Type: application/json" \
  -d '{
        "session_id": "12345",
        "message": "What is the weather today?",
        "context": {}
      }'

```

## Summary

- The Open-Notebook REST API accepts `model_override` in request payloads across chat and source-chat endpoints defined in [`api/routers/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/chat.py) and [`api/routers/source_chat.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/source_chat.py).
- Resolution follows a strict hierarchy: request-level overrides take precedence over session-level settings, which supersede global defaults from `DefaultModels`.
- The resolved model ID propagates to LangGraph via `RunnableConfig.configurable`, ensuring the correct LLM handles each specific invocation.
- Session updates persist to SurrealDB, allowing durable default model preferences for ongoing conversations.
- The implementation uses explicit null-checks in Python to distinguish between omitted parameters and intentional None values.

## Frequently Asked Questions

### What happens if I provide an invalid model ID in the override?

The request will proceed through the resolution logic, but the LangGraph execution will likely fail when the Esperanto provider attempts to instantiate the unrecognized model. Validation occurs at the provider level rather than the API boundary.

### Can I override the model for a single message without changing the session default?

Yes. By including `model_override` only in the `ExecuteChatRequest` payload, you affect that specific message's processing while leaving the underlying `ChatSession.model_override` unchanged in the database.

### Where does the system store the global default models?

The global defaults reside in the `DefaultModels` singleton class located in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py). This serves as the final fallback when neither request nor session specifies a model override.

### Does the source_chat router use the same override logic?

Yes. The [`api/routers/source_chat.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/source_chat.py) file implements identical request schema patterns and resolution logic, ensuring consistent model selection behavior across all conversational endpoints in the API.