# How to Use Model Override in LangGraph Configuration: Dynamic LLM Selection in Open Notebook

> Learn how to use model override in LangGraph configuration to dynamically select LLM models within your workflows. See how session state controls model selection for flexible AI applications.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-06-23

---

**You can override the LLM model in LangGraph workflows by setting `model_override` in the session state, which the graph checks when `configurable.model_id` is not provided in the RunnableConfig.**

The **open-notebook** repository implements a flexible model selection system for its LangGraph workflows, allowing developers to specify which LLM powers individual chat sessions. This article explains how to leverage `model_override` in LangGraph configuration to dynamically select models at the session level or per-invocation level.

## Understanding the Model Override Hierarchy

Open Notebook’s LangGraph pipelines resolve the target LLM through a two-tier fallback mechanism defined in [`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py) and [`open_notebook/graphs/source_chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/source_chat.py). The system checks two sources in order of priority:

1. **The `configurable` field of the LangGraph `RunnableConfig`** – passed directly during graph invocation
2. **The `model_override` attribute in session state** – persisted in the `ChatSession` domain object

The resolution logic follows this pattern:

```python

# open_notebook/graphs/chat.py (lines 34-36)

model_id = config.get("configurable", {}).get("model_id") \
           or state.get("model_override")

```

```python

# open_notebook/graphs/source_chat.py (lines 42-44)

config.get("configurable", {}).get("model_id") or state.get("model_override")

```

The **first non-null value** among these sources determines which model `provision_langchain_model()` instantiates for the workflow.

## Implementing Model Override in Open Notebook

### Session-Scoped Configuration via API

The most common approach sets a default model for an entire chat session. When you create or update a session via the FastAPI router, the system stores your preference in SurrealDB.

**Creating a new session with override:**

```bash
curl -X POST http://localhost:5055/chat/sessions \
  -H "Content-Type: application/json" \
  -d '{
        "title": "Research on AI safety",
        "model_override": "gpt-4o-mini"
      }'

```

The router in [`api/routers/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/chat.py) (lines 104-107) assigns this value to `session.model_override`, which persists in the database schema defined in [`open_notebook/domain/notebook.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/notebook.py) (lines 79-84).

**Updating an existing session:**

```bash
curl -X PATCH http://localhost:5055/chat/sessions/12345 \
  -H "Content-Type: application/json" \
  -d '{"model_override": "claude-3-5-sonnet"}'

```

This modifies the stored override (see [`api/routers/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/chat.py), lines 269-270), affecting all future graph invocations for that session.

### Runtime Configuration via RunnableConfig

For one-off model switches without modifying session state, pass the `model_id` directly in the `config` parameter during graph invocation:

```python
from open_notebook.graphs.chat import graph

# Override the session's default for this specific call only

result = await graph.ainvoke(
    state, 
    config={"configurable": {"model_id": "gemini-1.5-pro"}}
)

```

Because `configurable.model_id` takes precedence over `state["model_override"]`, this approach wins for the current execution while leaving the session default unchanged.

## Step-by-Step Implementation Examples

### Creating a Session with Model Override

When initiating a new conversation, explicitly set the model to control costs or capability levels:

```bash
curl -X POST http://localhost:5055/chat/sessions \
  -H "Content-Type: application/json" \
  -d '{
        "title": "Code Review Session",
        "model_override": "gpt-4-turbo"
      }'

```

Behind the scenes, the `ChatSession` object stores this value, making it available to `call_model_with_messages` during graph execution.

### Invoking the Graph Without Explicit Configuration

When you invoke the graph without specifying a `model_id` in the config, it automatically falls back to the session’s stored override:

```python
from open_notebook.graphs.chat import graph

state = {
    "messages": [],
    "notebook": None,
    "context": None,
    "model_override": None,  # Retrieved from database session

}

# Empty config triggers fallback to state["model_override"]

result = await graph.ainvoke(state, config={})

```

The graph reads the stored value from SurrealDB and provisions the corresponding LangChain model instance.

### Forcing a Different Model for a Single Call

To temporarily bypass the session default without updating the database record:

```python
result = await graph.ainvoke(
    state, 
    config={"configurable": {"model_id": "claude-3-opus-20240229"}}
)

```

This pattern is useful for A/B testing different models or handling specific message types that require particular capabilities.

## Key Source Files

Understanding the following files helps you trace how model override in LangGraph configuration propagates through the system:

- **[`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py)** – Core chat workflow containing the fallback logic for model selection (lines 30-36)
- **[`open_notebook/graphs/source_chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/source_chat.py)** – Source-specific chat implementation with identical override handling (lines 42-44)
- **[`open_notebook/domain/notebook.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/notebook.py)** – Defines the `ChatSession` model with the `model_override` field (lines 79-84)
- **[`api/routers/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/chat.py)** – REST endpoints for session creation and updates that propagate overrides to the database
- **[`api/routers/source_chat.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/source_chat.py)** – Corresponding endpoints for source-chat sessions

## Summary

- **Model override in LangGraph configuration** uses a two-tier priority system: `configurable.model_id` overrides `state["model_override"]`
- **Session-scoped overrides** persist in the `ChatSession` domain object and apply when no explicit config is provided
- **Per-call overrides** pass through the `RunnableConfig` without modifying stored session data
- The fallback chain ends with the default model defined in [`open_notebook/config.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/config.py) if neither source specifies a model
- All chat and source-chat graphs in the open-notebook repository implement this consistent override pattern

## Frequently Asked Questions

### What happens if both model_id and model_override are null?

If neither `configurable.model_id` nor `state["model_override"]` contains a value, the system falls back to the default model specified in the Esperanto configuration within [`open_notebook/config.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/config.py). This ensures the graph always has a valid LLM instance to execute.

### Can I change the model mid-conversation?

Yes. Send a PATCH request to `/chat/sessions/{id}` with a new `model_override` value, or update the field directly in SurrealDB. All subsequent graph invocations for that session will use the new model automatically, while previous messages retain their original processing context.

### Does model_override work