# LangGraph Workflow for Chat Operations in Open Notebook: Architecture and Implementation

> Explore the LangGraph workflow for chat in Open Notebook. Learn how this state machine leverages system prompts, model provisioning, and LLM invocation in FastAPI for seamless chat operations.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: architecture
- Published: 2026-06-27

---

**Open Notebook implements chat functionality as a LangGraph state machine that processes ThreadState through system prompt generation, model provisioning, and LLM invocation within a FastAPI backend.**

Open Notebook leverages LangGraph to power its conversational AI capabilities, creating a robust state-machine architecture for handling chat operations. The implementation, found in the [`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py) module, defines a deterministic workflow that transforms user inputs into AI responses through a five-step pipeline. This article examines the complete LangGraph workflow for chat operations in Open Notebook, breaking down how the system manages state, handles async operations, and generates context-aware responses.

## Understanding the LangGraph State Machine Architecture

### ThreadState Data Structure

The foundation of Open Notebook's chat system is the `ThreadState` class, defined as a `TypedDict` in [`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py). This state container tracks the complete conversation context:

```python
class ThreadState(TypedDict):
    messages: list  # Conversation history

    notebook: Optional[Any]  # Associated notebook context

    context: Optional[Any]  # Additional context data

    context_config: Optional[dict]  # Configuration settings

    model_override: Optional[str]  # Model selection override

```

### Graph Nodes and Flow Control

The LangGraph workflow consists of connected nodes that process the `ThreadState` sequentially. Each node receives the current state, performs a specific transformation, and returns modified state data for the next node to consume. The graph execution follows a linear path through the `agent` node, which orchestrates the LLM interaction.

## Step-by-Step LangGraph Chat Workflow Execution

### Step 1: Building the ThreadState

The workflow begins when the FastAPI backend instantiates a `ThreadState` containing the user's message history and any relevant notebook context. This initialization captures the `messages` list along with optional fields like `notebook`, `context`, `context_config`, and `model_override` to customize the AI behavior.

### Step 2: System Prompt Generation with Jinja Templates

When the graph reaches the `agent` node, the system generates a dynamic system prompt using the `ai_prompter.Prompter` class:

```python
system_prompt = Prompter(prompt_template="chat/system").render(data=state)

```

This invocation loads the `chat/system` Jinja template and renders it with the current `ThreadState` data, injecting notebook context and configuration parameters into the prompt.

### Step 3: Model Provisioning and Async Handling

Open Notebook provisions the LLM through `open_notebook.ai.provision.provision_langchain_model`. Because the LangGraph workflow runs synchronously (utilizing Sqlite-based checkpoints) while the model calls are async, the system wraps the provisioning in a new event loop:

```python

# Async handling to avoid blocking FastAPI event loop

run_in_new_loop()

# Or using ThreadPoolExecutor

concurrent.futures.ThreadPoolExecutor

```

This pattern ensures the FastAPI event loop remains responsive while the LangGraph state machine executes.

### Step 4: LLM Invocation and Message Processing

The provisioned LangChain model receives a payload containing the `SystemMessage` and user messages, then returns an `AIMessage`:

```python
ai_message = model.invoke(payload)

```

The `model.invoke()` method handles the actual communication with the underlying LLM provider, whether OpenAI, Anthropic, or other supported backends.

### Step 5: Response Cleaning and State Updates

After receiving the raw AI response, the workflow cleans the output by stripping extraneous formatting and thought tokens (such as `<think>` tags or reasoning steps). The cleaned message is then appended to the `messages` list in the `ThreadState`, preparing the state for the next conversation turn or graph invocation.

## Implementation Details in open_notebook/graphs/chat.py

The complete workflow is orchestrated through the [`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py) file, which defines the node functions and graph compilation. The synchronous nature of LangGraph's checkpointing requires special handling for async LLM operations, implemented through thread pool executors and new event loops to maintain compatibility with FastAPI's async architecture.

Key implementation characteristics:

- **State Persistence**: Sqlite-based checkpoints ensure conversation continuity across API calls
- **Template System**: Jinja2 templates in `chat/system` allow customizable system prompts without code changes
- **Model Flexibility**: The `model_override` field in `ThreadState` enables dynamic model selection per conversation

## Summary

- Open Notebook implements chat operations as a LangGraph state machine in [`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py)
- The `ThreadState` TypedDict maintains conversation context including messages, notebook data, and configuration
- System prompts are generated using `ai_prompter.Prompter` with the `chat/system` Jinja template
- Async model provisioning uses `run_in_new_loop()` or `ThreadPoolExecutor` to prevent FastAPI event loop blocking
- The workflow follows a five-step pipeline: state building, prompt generation, model provisioning, LLM invocation, and response cleaning

## Frequently Asked Questions

### What is ThreadState in Open Notebook's LangGraph implementation?

`ThreadState` is a `TypedDict` defined in [`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py) that serves as the state container for the LangGraph workflow. It tracks the `messages` list, optional `notebook` context, `context_config` settings, and `model_override` preferences, allowing the graph to maintain conversation history and contextual awareness across multiple turns.

### How does Open Notebook handle async operations in LangGraph?

Since LangGraph runs synchronously with Sqlite-based checkpoints, Open Notebook wraps asynchronous model calls in `run_in_new_loop()` or executes them via `concurrent.futures.ThreadPoolExecutor`. This approach prevents the synchronous LangGraph execution from blocking the FastAPI event loop while still allowing async LLM provisioning through `open_notebook.ai.provision.provision_langchain_model`.

### Where is the system prompt template defined in Open Notebook?

The system prompt template is defined as a Jinja2 template named `chat/system`, which is rendered by the `ai_prompter.Prompter` class when the graph reaches the agent node. The template receives the entire `ThreadState` as data, enabling dynamic injection of notebook context and conversation history into the system prompt.

### Can I override the model selection in Open Notebook's chat workflow?

Yes, the `ThreadState` includes an optional `model_override` field that allows per-conversation model selection. When provided, this field influences the model provisioning step in [`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py), enabling specific conversations to use different LLM providers or model versions than the default configuration.