# How to Override AI Models per Request Using RunnableConfig in LangGraph

> Learn to override AI models per request in LangGraph using RunnableConfig. Dynamically change LLMs for specific API calls without altering global configurations. Maximize flexibility and control.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-07-05

---

**You can override AI models for individual requests in LangGraph by passing a `model_id` through the `configurable` dictionary inside `RunnableConfig`, which flows from the API layer through graph nodes to the model provisioning helper without affecting global settings.**

Open Notebook uses LangGraph to orchestrate AI-driven workflows, and you can dynamically select which language model processes each request by leveraging the `RunnableConfig` object. This approach allows you to specify a `model_id` for a single invocation while keeping the rest of your application configuration intact.

## How RunnableConfig Enables Per-Request Model Overrides

LangGraph's `RunnableConfig` object provides a `configurable` dictionary that travels with every graph invocation. In `lfnovo/open-notebook`, this dictionary carries two critical pieces of metadata: the `thread_id` for state isolation and the `model_id` for model selection. Because `RunnableConfig` is thread-local and request-scoped, you can override the default model for a single execution without side effects on concurrent requests or global settings.

## The Three-Layer Override Flow

The model override travels through three distinct layers, from the HTTP API down to the model instantiation logic.

### API Layer: Capturing the Model Override

In [`api/routers/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/chat.py), the FastAPI router accepts an optional `model_override` field in the request body. The `ExecuteChatRequest` model defines this field, and the route handler packages it into a `RunnableConfig`:

```python
class ExecuteChatRequest(BaseModel):
    session_id: str
    message: str
    context: Dict[str, Any]
    model_override: Optional[str] = None   # Per-request model selection

@router.post("/chat/execute", response_model=ExecuteChatResponse)
async def execute_chat(req: ExecuteChatRequest):
    cfg = RunnableConfig(
        configurable={
            "thread_id": f"chat_session:{req.session_id}",
            "model_id": req.model_override,  # Passed downstream to nodes

        }
    )
    result = await chat_graph.ainvoke(
        {"messages": [...], "context": req.context},
        config=cfg,
    )

```

### Graph Layer: Extracting Configuration in Nodes

Each node in [`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py) receives the `RunnableConfig` as a second argument. The `call_model_with_messages` function extracts the `model_id` from the configurable dictionary and passes it to the provisioning helper:

```python
def call_model_with_messages(state: ThreadState, config: RunnableConfig) -> dict:
    # Pull the optional override supplied by the API

    model_id = config.get("configurable", {}).get("model_id") \
                or state.get("model_override")
    
    # Provision the concrete LangChain model

    model = await provision_langchain_model(
        str(payload), model_id, "chat", max_tokens=8192
    )
    ai_message = model.invoke(payload)
    return {"messages": [ai_message]}

```

### Provisioning Layer: Resolving the Model Instance

The `provision_langchain_model` function in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) makes the final decision. If a `model_id` is present in the configuration, it loads that specific model; otherwise, it falls back to the default model for the requested type:

```python
async def provision_langchain_model(content, model_id, default_type, **kwargs):
    if model_id:
        # Explicit request – load that model

        model = await model_manager.get_model(model_id, **kwargs)
    else:
        # No override – use the default for the type (chat, embed, etc.)

        model = await model_manager.get_default_model(default_type, **kwargs)
    
    return model.to_langchain()

```

## Complete Implementation Example

Here is the end-to-end flow showing how to trigger a model override from a client request:

```bash
curl -X POST https://api.example.com/chat/execute \
  -H "Content-Type: application/json" \
  -d '{
        "session_id": "12345",
        "message": "Explain quantum tunneling",
        "context": { "sources": [] },
        "model_override": "gpt-4o-mini"
      }'

```

When this request hits the graph, the `call_model_with_messages` node will use `gpt-4o-mini` instead of the default chat model. You can apply this same pattern to other graphs like [`open_notebook/graphs/ask.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/ask.py) or [`open_notebook/graphs/source_chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/source_chat.py) by reading the same `configurable` key in their node functions.

## Practical Use Cases for Model Overrides

**Testing new LLMs**: Send `model_override: "gpt-4o-mini"` in the request JSON to evaluate a new model's behavior without changing the application-wide configuration.

**Handling large contexts**: Force a specific long-context model (such as `"anthropic/claude-3-5-sonnet"`) only for requests that exceed a certain token threshold, while keeping standard queries on cheaper models.

**Provider-specific features**: Pass provider-specific model IDs to access unique capabilities (like extended thinking modes or specific tool-calling formats) for experimental features.

## Summary

- **RunnableConfig** carries a `configurable` dictionary that includes `thread_id` for state isolation and `model_id` for model selection.
- The **API layer** in [`api/routers/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/chat.py) accepts an optional `model_override` field and packages it into the configuration.
- **Graph nodes** in [`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py) extract the `model_id` from `config.get("configurable", {})` and pass it to the provisioning helper.
- The **provisioning layer** in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) resolves the explicit `model_id` or falls back to defaults, ensuring no global side effects.
- This pattern works across all LangGraph workflows in the repository, including [`source_chat.py`](https://github.com/lfnovo/open-notebook/blob/main/source_chat.py) and [`transformation.py`](https://github.com/lfnovo/open-notebook/blob/main/transformation.py).

## Frequently Asked Questions

### What is RunnableConfig in LangGraph?

`RunnableConfig` is a configuration object that LangGraph passes to every node during graph execution. It contains a `configurable` dictionary that can hold arbitrary key-value pairs, allowing you to inject request-specific data like `thread_id` for state management or `model_id` for model selection without modifying global state.

### How does Open Notebook isolate state between chat sessions?

Open Notebook uses the `thread_id` key inside the `configurable` dictionary to isolate LangGraph state. Each chat session receives a unique `thread_id` (formatted as `chat_session:{session_id}`), ensuring that conversation history and context remain separate across different users and sessions.

### Can I use model overrides in other graphs besides chat?

Yes. The same pattern applies to [`open_notebook/graphs/ask.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/ask.py), [`open_notebook/graphs/source_chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/source_chat.py), and [`open_notebook/graphs/transformation.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/transformation.py). Any node that receives `RunnableConfig` can extract `model_id` from `config.get("configurable", {})` and pass it to `provision_langchain_model` to override the default model for that specific execution.

### What happens if the specified model_id is not found?

If the `model_id` provided in the override does not exist in the model manager, the `provision_langchain_model` function in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) will attempt to load it via `model_manager.get_model(model_id)`. If that fails, the system will raise an error rather than silently falling back to the default, ensuring explicit behavior and preventing unexpected model switches.