How LangGraph Orchestrates Chat, Ask, and Source Operations in Open Notebook

Open Notebook uses LangGraph to define its chat, ask, and source flows as stateful, checkpointed graphs where typed state schemas, registered nodes, and conditional edges manage LLM calls and context retrieval.

The lfnovo/open-notebook repository leverages LangGraph to power its three core interactive experiences: plain chat, search-backed ask, and source-oriented chat. Each flow is implemented as a dedicated StateGraph that coordinates model provisioning, prompt rendering, and context assembly through a common pattern of typed state dictionaries and SQLite persistence. Understanding how LangGraph orchestrates chat, ask, and source operations reveals a declarative, extensible architecture that is easy to trace and modify.

Common LangGraph Architecture in Open Notebook

StateGraph and Typed State Schemas

Every graph begins with a state schema defined as a TypedDict. The plain chat and ask flows share a ThreadState that tracks messages, notebooks, and intermediate results, while the source chat flow uses SourceChatState to hold source_id, Source objects, and SourceInsight data. Nodes registered via add_node(name, callable) read and write these state fields during execution.

Checkpointing and Resumability

Runtime state is persisted through SqliteSaver, which writes graph checkpoints to the path specified by LANGGRAPH_CHECKPOINT_FILE. This ensures that every conversational turn is recoverable and that the graph can resume from its last known state across sessions.

Shared Node Utilities

Nodes rely on two central helpers. provision_langchain_model (located in open_notebook/ai/provision.py) selects the provider through Esperanto, applies model overrides from graph config, and returns a LangChain-compatible instance. Prompter renders Jinja2 templates such as chat/system, ask/entry, and source_chat/system with the current state payload.

Plain Chat Graph

Located in open_notebook/graphs/chat.py, the plain chat graph is the simplest of the three flows.

Single-Node Design with System Prompting

The graph defines one node: call_model_with_messages. This node builds a system prompt using the chat/system template, provisions a model via provision_langchain_model, invokes it with the message list from ThreadState, and strips any LLM "thinking" tags from the output. Graph wiring is linear: START → agent → END.

Invoking the Chat Graph

You can run the chat graph asynchronously by passing a message list and an optional notebook object:

from open_notebook.graphs.chat import graph as chat_graph

await chat_graph.ainvoke(
    {
        "messages": [],
        "notebook": my_notebook,
    },
    config={"configurable": {"model_id": "gpt-4o"}}
)

Ask Graph with Conditional Branching

The ask flow in open_notebook/graphs/ask.py demonstrates LangGraph’s ability to spawn dynamic sub-graphs. It answers a user question by generating a search strategy, executing parallel retrievals, and synthesizing a final response.

Strategy Generation

The first node, call_model_with_messages, uses the ask/entry prompt to generate a JSON-structured Strategy object. This strategy contains a list of Search terms that the system must investigate before answering.

Dynamic Search and Answer Nodes

After the strategy is produced, an add_conditional_edges branch evaluates the search list and dispatches each term to the provide_answer node via LangGraph's Send mechanism. The provide_answer node executes vector_search for its assigned term, then calls a second LLM with the ask/query_process prompt to synthesize an intermediate answer. Once all parallel searches complete, the write_final_answer node gathers every intermediate result and prompts a final model (ask/final_answer) to produce the concise response.

Invoking the Ask Graph

Configuration keys let you assign different models to each phase:

from open_notebook.graphs.ask import graph as ask_graph

await ask_graph.ainvoke(
    {"question": "What are the privacy guarantees of Open Notebook?"},
    config={
        "configurable": {
            "strategy_model": "claude-3",
            "answer_model": "gpt-4o-mini",
            "final_answer_model": "gpt-4o"
        }
    },
)

Source-Oriented Chat Graph

The source chat flow in open_notebook/graphs/source_chat.py grounds the LLM in a specific document rather than general conversation.

Context Enrichment with ContextBuilder

The core node is call_model_with_source_context. It delegates to ContextBuilder (in open_notebook/utils/context_builder.py) to assemble the source's full text, its insights, and metadata, capping the context at roughly 50,000 tokens. The node then renders the source_chat/system Jinja template with this assembled data, provisions a chat model (with an optional model override), and invokes the LLM. The enriched context, including Source and SourceInsight objects, travels with the state.

Invoking the Source Chat Graph

Like the plain chat graph, wiring is linear: START → source_chat_agent → END:

from open_notebook.graphs.source_chat import source_chat_graph

await source_chat_graph.ainvoke(
    {
        "messages": [],
        "source_id": "src_123",
    },
    config={"configurable": {"model_id": "gpt-4o"}}
)

Summary

• StateGraph declaration – Each flow uses a dedicated StateGraph with a strongly typed state (ThreadState or SourceChatState) to manage conversational context.
• Linear vs. conditional wiring – Plain and source chat use single-node linear edges, while the ask graph leverages add_conditional_edges and Send to execute dynamic, parallel search subgraphs.
• Persistence – All graphs rely on SqliteSaver writing to LANGGRAPH_CHECKPOINT_FILE for resumable execution.
• Shared infrastructure – provision_langchain_model, Prompter, and ContextBuilder centralize model setup, prompt rendering, and source-specific context assembly across open_notebook/graphs/chat.py, open_notebook/graphs/ask.py, and open_notebook/graphs/source_chat.py.

Frequently Asked Questions

What is the role of SqliteSaver in Open Notebook's LangGraph flows?

SqliteSaver persists graph checkpoints to the SQLite file defined by LANGGRAPH_CHECKPOINT_FILE. According to the lfnovo/open-notebook source code, this allows the chat, ask, and source chat graphs to resume execution across turns and sessions without losing intermediate state.

How does the ask graph decide how many searches to run?

The ask graph's first node generates a JSON Strategy containing a list of Search objects. The graph then uses add_conditional_edges to map each search item into a Send to the provide_answer node, dynamically spawning one sub-graph per search term before aggregating results in write_final_answer.

Can I use different models for each phase of the ask flow?

Yes. The ask graph accepts separate configuration keys under configurable: strategy_model for planning, answer_model for intermediate retrieval, and final_answer_model for synthesis. provision_langchain_model in open_notebook/ai/provision.py resolves each override at runtime.

What makes the source chat graph different from plain chat?

The source chat graph in open_notebook/graphs/source_chat.py uses ContextBuilder to inject a specific source's full text, insights, and metadata into the prompt via the source_chat/system template. This restricts the LLM's context to the referenced source, whereas the plain chat in open_notebook/graphs/chat.py relies on general system prompts and message history.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}