Understanding the ask.py Graph Flow: Multi-Search Strategy and Answer Synthesis in Open Notebook

The ask.py graph implements a LangGraph-powered workflow that dynamically generates search strategies, executes parallel vector searches, and synthesizes intermediate results into a coherent final answer.

The ask.py module in the lfnovo/open-notebook repository orchestrates a sophisticated multi-search workflow using LangGraph's StateGraph. This system enables the LLM to first devise a search strategy, then execute multiple independent vector lookups, and finally combine the results into a single synthesized response. Understanding this graph flow is essential for developers looking to customize the question-answering pipeline or debug the orchestration logic.

Core Graph Architecture

The graph consists of four specialized nodes defined in open_notebook/graphs/ask.py that transform a user question into a final answer through a planned, multi-step retrieval process.

Strategy Generation Node (agent)

The agent node (lines 51-75) serves as the entry point where the LLM generates a structured search plan. The call_model_with_messages function reads the user prompt, renders the ask/entry system prompt, and calls a provisioned LLM to produce a JSON output. This output is parsed into a Strategy Pydantic model containing a list of Search items, each defined by a term and specific instructions.

Dynamic Query Dispatch (trigger_queries)

Rather than hard-coding parallel branches, the trigger_queries function (lines 83-95) creates a dynamic list of Send-type edges at runtime. For every Search object in state["strategy"].searches, it dispatches one provide_answer node. This architecture supports up to five concurrent searches without modifying the graph structure, allowing the LLM to decide exactly how many queries are needed based on the question complexity.

Per-Search Answer Generation (provide_answer)

Each dispatched provide_answer node (lines 98-124) executes a self-contained retrieval and reasoning cycle. The node first calls vector_search (implemented in open_notebook/domain/notebook.py) to perform a similarity lookup on the search term. It then builds the ask/query_process prompt with the retrieved document IDs, sends this to the answer LLM, and returns a cleaned intermediate answer that gets accumulated in state["answers"].

Final Answer Synthesis (write_final_answer)

The write_final_answer node (lines 126-144) aggregates all intermediate answers from the parallel search branches. It renders the ask/final_answer prompt with the complete context, invokes the final-answer LLM, and produces the user-facing response. This node only executes after all provide_answer instances have completed, ensuring the synthesis has access to the full information spectrum.

Step-by-Step Execution Flow

The graph follows a deterministic sequence from invocation to completion:

1. Graph Entry — The workflow starts at START → agent with an initial state containing the user question.

2. Strategy Formulation — The agent node generates a JSON strategy describing the search approach (e.g., splitting a complex query into architectural and policy components).

3. Dynamic Dispatch — The trigger_queries function inspects state["strategy"] and creates parallel Send("provide_answer", ...) edges for each search term.

4. Parallel Retrieval — Multiple provide_answer instances run concurrently, each performing vector lookups in SurrealDB and generating sub-answers via the ask/query_process prompt.

5. Aggregation and Synthesis — Once all branches complete, write_final_answer combines the results using the ask/final_answer prompt to produce a coherent final response.

6. Completion — The graph reaches END (lines 146-155), returning final_answer to the caller.

Implementation Code Examples

Invoking the Graph from Python

You can execute the graph asynchronously using the compiled StateGraph object:

from open_notebook.graphs.ask import graph

# Initialize with the user question

payload = {"question": "What are the privacy features of Open Notebook?"}

# Execute the workflow (typically via FastAPI)

final_output = await graph.ainvoke(payload, config={})
print(final_output["final_answer"])

The graph object is instantiated at the bottom of ask.py (lines 146-155) and represents the complete compiled workflow.

Strategy JSON Structure

When the LLM processes the ask/entry prompt, it returns a structured JSON object that maps to the Strategy model:

{
  "reasoning": "The user asks about privacy; we need both architectural and policy info.",
  "searches": [
    {
      "term": "privacy architecture",
      "instructions": "Summarize the high-level design and data-flow controls."
    },
    {
      "term": "privacy policy",
      "instructions": "List the key privacy statements from the repository."
    }
  ]
}

These Search objects drive the dynamic dispatch mechanism, with each item spawning an independent provide_answer node.

Configuring LLM Models

The graph supports different models for each phase via the configurable parameter:

config = {
    "configurable": {
        "strategy_model": "gpt-4o-mini",      # For initial search planning

        "answer_model": "gpt-4o-mini",        # For each sub-search answer

        "final_answer_model": "gpt-4o"        # For the final synthesis

    }
}

final_output = await graph.ainvoke(payload, config=config)

The provision_langchain_model helper (defined in open_notebook/ai/provision.py) reads these configuration values to instantiate the appropriate LangChain wrappers for each node.

Key Implementation Files

The multi-search strategy relies on several supporting modules:

• open_notebook/graphs/ask.py — Core graph definition, node implementations, and LangGraph wiring (lines 146-155).
• open_notebook/ai/provision.py — Supplies provisioned LLMs via provision_langchain_model used by all graph nodes.
• open_notebook/domain/notebook.py — Implements vector_search for retrieving embeddings from SurrealDB.
• ai_prompter/prompter.py — Renders the three prompt templates: ask/entry, ask/query_process, and ask/final_answer.
• open_notebook/utils/*.py — Utility functions like clean_thinking_content and extract_text_content for sanitizing LLM outputs.

Summary

• The ask.py graph flow uses LangGraph to orchestrate a multi-step retrieval and synthesis pipeline.
• The agent node generates a dynamic Strategy that determines which searches to execute.
• Dynamic dispatch via Send edges allows parallel execution of up to five independent vector searches.
• Each provide_answer node performs a targeted vector lookup and generates an intermediate answer.
• The write_final_answer node synthesizes all intermediate results into a final coherent response using configurable LLM models.

Frequently Asked Questions

How does the graph handle multiple search queries without hard-coded branches?

The trigger_queries function (lines 83-95) creates dynamic Send edges at runtime based on the Strategy model output. This allows the graph to spawn exactly as many provide_answer nodes as needed (up to five) without modifying the underlying graph structure, providing flexibility for complex questions requiring multiple retrieval angles.

What determines which LLM model is used for answer synthesis?

The provision_langchain_model helper reads from the configurable dictionary in the graph config. You can specify strategy_model, answer_model, and final_answer_model independently, allowing cost-effective models for planning and retrieval while using more capable models for the final synthesis step.

Where is the vector search actually implemented?

The vector_search function is defined in open_notebook/domain/notebook.py. The provide_answer node calls this function to perform similarity searches against SurrealDB embeddings, then passes the retrieved document IDs to the LLM via the ask/query_process prompt template.

Can the graph execution be customized or extended?

Yes, the modular node structure in ask.py allows developers to override specific behaviors. You can modify the prompt templates in ai_prompter/prompter.py, swap the vector search implementation, or adjust the Strategy Pydantic model to include additional metadata fields that influence downstream node behavior.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}