LangGraph Workflow for Content Transformations in Open Notebook: Implementation Guide

Open Notebook orchestrates AI-powered content transformations through a LangGraph StateGraph that processes text via LLM prompts, cleans the generated output, and persists results as source insights.

Open Notebook leverages LangGraph to manage its content transformation pipeline. This system accepts either raw input text or existing Source records, routes them through configurable language model prompts, and stores structured outputs back to the database. The implementation centers on a state-driven graph defined in open_notebook/graphs/transformation.py that coordinates prompt construction, model provisioning via the Esperanto multi-model stack, and result persistence.

Architecture Overview

The transformation workflow relies on a minimal but robust state machine built on LangGraph primitives. The architecture decouples the graph definition from the execution logic, allowing both API-driven and programmatic invocation.

Core Components

• TransformationState: A TypedDict that carries input_text, source, transformation, and the final output across graph nodes.
• StateGraph: A LangGraph StateGraph initialized with TransformationState as its schema, containing a single node named "agent" that executes the run_transformation function.
• run_transformation: An async node function that builds prompts, provisions models, invokes the LLM chain, cleans responses, and writes insights back to sources.
• DefaultPrompts: An optional global configuration record (open_notebook:default_prompts) containing transformation instructions that prepend to user-defined prompts.

The Transformation Execution Flow

The workflow executes in a linear sequence from graph entry to completion, with explicit error handling at each stage.

Graph Initialization

The compiled graph uses a single-node architecture with explicit start and end edges:

from langgraph.graph import StateGraph, START, END

agent_state = StateGraph(TransformationState)
agent_state.add_node("agent", run_transformation)
agent_state.add_edge(START, "agent")
agent_state.add_edge("agent", END)
graph = agent_state.compile()

This structure lives in open_notebook/graphs/transformation.py and exposes the compiled graph object for direct invocation.

Input Resolution

The run_transformation node receives a state dictionary containing the transformation definition. If input_text is missing from the state, the function automatically falls back to source.full_text, enabling operations on both custom snippets and existing Source records.

Prompt Construction

The system constructs the final prompt through a layered approach:

1. Base prompt: Retrieved from the Transformation domain model's prompt field.
2. Global instructions: If a DefaultPrompts record exists with transformation_instructions, these prepend to the base prompt.
3. Input marker: The combined text ends with a # INPUT marker before the actual content.

The assembled prompt renders through the ai-prompter Prompter class to produce the final system message.

Model Provisioning

The workflow provisions the appropriate LLM using the Esperanto multi-model abstraction layer:

chain = await provision_langchain_model(
    str(payload),
    config.get("configurable", {}).get("model_id"),
    "transformation",
    max_tokens=8192,
)

This helper selects providers (OpenAI, Anthropic, etc.) based on the model_id passed via configurable graph state.

LLM Invocation and Response Processing

The chain receives a payload containing a SystemMessage (the rendered prompt) and a HumanMessage (the actual content). After invocation:

1. Content extraction: extract_text_content strips non-text artifacts from the raw response.
2. Thinking content removal: clean_thinking_content eliminates chain-of-thought prefixes that models may emit.

Persistence and Error Handling

If a Source object exists in state, the cleaned output stores as an insight under the transformation's title:

await source.add_insight(transformation.title, cleaned_content)

The node returns a dictionary with the key output containing the final text, which becomes the graph's terminal state.

For error handling, the system distinguishes between domain-specific OpenNotebookError instances (re-raised unchanged) and generic exceptions. The latter classify through classify_error and re-throw as user-friendly error classes.

API Service Layer

The service facade in api/transformations_service.py exposes CRUD operations and an execution endpoint that bridges HTTP requests to the graph workflow.

Key Service Methods

Method	Purpose
get_all_transformations	Fetches transformation records and builds Transformation domain objects.
create_transformation	Persists new transformation definitions to the database.
update_transformation	Applies PATCH-style updates to existing transformations.
delete_transformation	Removes transformations by ID.
execute_transformation	Invokes api_client.execute_transformation(transformation_id, input_text, model_id) to trigger the LangGraph workflow.

The execution method translates HTTP parameters into the graph's TransformationState and delegates to the compiled graph via the API client.

Practical Implementation Examples

Defining a Transformation

Create domain models using the Transformation class and persist via the service layer:

from open_notebook.domain.transformation import Transformation

my_transform = Transformation(
    name="summarize",
    title="Summarize Article",
    description="Creates a concise summary of the given article.",
    prompt="""
        You are a concise writer. Summarize the following text in 3 short paragraphs.
    """,
    apply_default=False,
)

# Persist via the service

transformations_service.create_transformation(**my_transform.dict())

The domain model definition resides in open_notebook/domain/transformation.py.

Executing via the Service Layer

Invoke transformations through the high-level service interface:

result = transformations_service.execute_transformation(
    transformation_id="01F7ZQ3V4K",
    input_text="Long article text …",
    model_id="openai:gpt-4o-mini"
)

print(result["output"])

This call internally builds the graph state, runs the run_transformation node, and returns the cleaned output without exposing graph complexity.

Direct Graph Invocation

For background jobs or custom workflows, bypass the HTTP layer and invoke the compiled graph directly:

from open_notebook.graphs.transformation import graph, TransformationState

state: TransformationState = {
    "input_text": "Raw document …",
    "source": None,
    "transformation": my_transform,
    "output": ""
}

final_state = await graph.ainvoke(
    state,
    config={"configurable": {"model_id": "anthropic:claude-3-5-sonnet"}}
)

print(final_state["output"])

This approach provides full control over the TransformationState and configurable model selection.

Summary

• LangGraph StateGraph powers Open Notebook's content transformation pipeline through a single-node workflow defined in open_notebook/graphs/transformation.py.
• TransformationState carries input text, source references, and transformation definitions across the graph execution.
• run_transformation handles the complete lifecycle: prompt building (with optional DefaultPrompts prepending), model provisioning via provision_langchain_model, LLM invocation, and insight persistence via source.add_insight.
• Error classification distinguishes domain-specific errors from generic exceptions, ensuring user-friendly error messages.
• API Service Layer in api/transformations_service.py provides CRUD operations and execution endpoints that translate HTTP requests into graph invocations.

Frequently Asked Questions

How does the LangGraph workflow handle missing input text?

The run_transformation node automatically falls back to source.full_text when input_text is absent from the state. This design allows the graph to operate on both custom text snippets passed directly and existing Source records loaded from the database.

Can I use different language models for different transformations?

Yes. The graph accepts a model_id through the configurable graph state (config["configurable"]["model_id"]). Valid identifiers include provider-prefixed strings like "openai:gpt-4o-mini" or "anthropic:claude-3-5-sonnet", which the provision_langchain_model function maps to the appropriate Esperanto provider.

What happens when the LLM returns chain-of-thought content?

The response cleaning pipeline in open_notebook/utils/text_utils.py processes raw LLM output through extract_text_content and clean_thinking_content. These functions strip non-text artifacts and remove thinking prefixes (such as model reasoning blocks) before persisting the final content as a source insight.

How do I add global instructions to all transformations?

Create a DefaultPrompts record with the key open_notebook:default_prompts and populate the transformation_instructions field. When apply_default=True on a transformation, these instructions automatically prepend to the transformation-specific prompt before the # INPUT marker.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}