How the Open-Notebook REST API Handles Model Configuration Overrides Per Request

The Open-Notebook REST API implements a three-tier fallback system that prioritizes request-level model overrides, falls back to session-specific configurations stored in SurrealDB, and finally defaults to global settings, injecting the resolved model ID directly into the LangGraph execution context.

The open-notebook repository provides a flexible REST API for conversational AI interactions that allows dynamic LLM selection without restarting services or modifying environment variables. By supporting per-request model configuration overrides, the system enables developers to switch between GPT-4, Claude, or other providers on a per-message basis while maintaining persistent session preferences. This architecture ensures that individual requests can specify exactly which model should handle the inference, with clear precedence rules determining the final selection.

Where the API Accepts Model Overrides

Chat Endpoints

In api/routers/chat.py, the Pydantic request models define an optional model_override field across all relevant operations. The CreateSessionRequest, UpdateSessionRequest, and ExecuteChatRequest schemas each accept this parameter, allowing clients to specify a model identifier such as "gpt-4o" or "claude-3-opus" at different stages of the conversation lifecycle.

Source Chat Endpoints

The same pattern appears in api/routers/source_chat.py, ensuring consistent behavior across different conversational interfaces. Whether initiating a new session or executing a message against an existing one, the API surface accepts the override uniformly.

The Three-Tier Resolution Order

The resolution logic follows a strict precedence hierarchy to determine the effective model:

1. Request-level override – The explicit model_override value provided in the current HTTP request takes highest priority.
2. Session-level override – If the request omits the override, the system checks the session.model_override attribute stored in the database.
3. Global default – When neither request nor session specifies a model, the application falls back to the DefaultModels singleton accessed via open_notebook.ai.models.DefaultModels.

This cascade ensures maximum flexibility while maintaining sensible defaults.

Propagation to LangGraph Execution

Once resolved, the model identifier propagates through the execution stack. In open_notebook/graphs/chat.py, the system injects the model_override into the LangGraph invocation via the RunnableConfig configurable dictionary:

result = chat_graph.invoke(
    input=state_values,
    config=RunnableConfig(
        configurable={"thread_id": full_session_id, "model_id": model_override},
    ),
)

The Esperanto provider receives this model_id and instantiates the appropriate LLM client for that specific call, enabling true per-request model switching without thread contamination.

Persistence and Session Management

Updates to a session's default model persist to SurrealDB. When calling PUT /chat/sessions/{session_id}, the UpdateSessionRequest payload writes the model_override value directly to the ChatSession.model_override field in open_notebook/domain/notebook.py. Subsequent requests that omit an explicit override automatically inherit this stored preference.

Implementation Details

The resolution logic in api/routers/chat.py implements the fallback chain explicitly:

model_override = (
    request.model_override
    if request.model_override is not None
    else getattr(session, "model_override", None)
)

The resolved value then populates the state dictionary passed to the graph:

state_values["model_override"] = model_override

Practical API Examples

Create a session with a specific default model:

curl -X POST http://localhost:5055/chat/sessions \
  -H "Content-Type: application/json" \
  -d '{
        "notebook_id": "my-notebook",
        "title": "Demo Session",
        "model_override": "gpt-4o"
      }'

Execute a message overriding the session default:

curl -X POST http://localhost:5055/chat/execute \
  -H "Content-Type: application/json" \
  -d '{
        "session_id": "12345",
        "message": "Explain quantum computing",
        "model_override": "claude-3-opus"
      }'

Update an existing session's default:

curl -X PUT http://localhost:5055/chat/sessions/12345 \
  -H "Content-Type: application/json" \
  -d '{
        "model_override": "gpt-4"
      }'

Rely on automatic fallback:

curl -X POST http://localhost:5055/chat/execute \
  -H "Content-Type: application/json" \
  -d '{
        "session_id": "12345",
        "message": "What is the weather today?",
        "context": {}
      }'

Summary

• The Open-Notebook REST API accepts model_override in request payloads across chat and source-chat endpoints defined in api/routers/chat.py and api/routers/source_chat.py.
• Resolution follows a strict hierarchy: request-level overrides take precedence over session-level settings, which supersede global defaults from DefaultModels.
• The resolved model ID propagates to LangGraph via RunnableConfig.configurable, ensuring the correct LLM handles each specific invocation.
• Session updates persist to SurrealDB, allowing durable default model preferences for ongoing conversations.
• The implementation uses explicit null-checks in Python to distinguish between omitted parameters and intentional None values.

Frequently Asked Questions

What happens if I provide an invalid model ID in the override?

The request will proceed through the resolution logic, but the LangGraph execution will likely fail when the Esperanto provider attempts to instantiate the unrecognized model. Validation occurs at the provider level rather than the API boundary.

Can I override the model for a single message without changing the session default?

Yes. By including model_override only in the ExecuteChatRequest payload, you affect that specific message's processing while leaving the underlying ChatSession.model_override unchanged in the database.

Where does the system store the global default models?

The global defaults reside in the DefaultModels singleton class located in open_notebook/ai/models.py. This serves as the final fallback when neither request nor session specifies a model override.

Does the source_chat router use the same override logic?

Yes. The api/routers/source_chat.py file implements identical request schema patterns and resolution logic, ensuring consistent model selection behavior across all conversational endpoints in the API.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}