# How LangGraph Transformations Process Source Content in Open-Notebook

> Discover how Open-Notebook uses LangGraph transformations to process source content. Learn how it extracts text, applies prompts, invokes LLMs, and saves cleaned insights for your projects.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: deep-dive
- Published: 2026-06-15

---

**Open-Notebook implements content transformations as a compiled LangGraph workflow that extracts source text, applies customizable prompt templates, invokes provisioned LLMs, and persists cleaned results as source insights.**

Open-Notebook leverages LangGraph to orchestrate content-wise transformations through a compact, deterministic workflow defined in [`open_notebook/graphs/transformation.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/transformation.py). This architecture processes raw source material—whether passed directly as text or retrieved from persisted records—through a standardized pipeline of prompt construction, model invocation, and result persistence. Understanding how these LangGraph transformations handle source content reveals the system's model-agnostic approach to AI-powered document analysis.

## Defining the Transformation State

The workflow relies on `TransformationState`, a TypedDict defined in [`open_notebook/graphs/transformation.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/transformation.py) that tracks the complete execution context. This state holds four critical components:

- **`input_text`** – The raw text submitted for transformation (optional if a `Source` is provided).
- **`source`** – A `Source` object containing the full extracted text and metadata.
- **`transformation`** – The `Transformation` definition including the prompt template and title.
- **`output`** – The final cleaned result produced by the LLM.

This state structure ensures that the graph can operate in two modes: processing ad-hoc text supplied directly, or processing persisted content from the notebook's source repository.

## The Transformation Execution Flow

The LangGraph workflow consists of a single node (`run_transformation`) that executes a six-step pipeline from text ingestion to insight persistence.

### Text Retrieval and State Preparation

The `run_transformation` node first extracts the `Source` object and input content from the state. If `input_text` is absent, the node automatically falls back to `source.full_text`—the complete extracted text of the persisted source record. This fallback mechanism guarantees that the workflow functions whether the caller supplies temporary text or references a stored document.

### Prompt Construction and Model Invocation

Once the raw content is secured, the system constructs the final prompt through three layers:

1. **Default Instructions**: The system retrieves global default prompts from `DefaultPrompts` (defined in [`open_notebook/domain/transformation.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/transformation.py)) and optionally prefixes them to the transformation-specific template.
2. **Template Rendering**: The `transformation.prompt` template is rendered with the current state using `ai_prompter.Prompter`, producing the final instruction set.
3. **Message Construction**: The rendered prompt is wrapped in a `SystemMessage`, while the source content is sent as a `HumanMessage`.

The provisioned model is instantiated via `provision_langchain_model` (located in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py)), which builds a LangChain chain targeting the model ID specified in the LangGraph config (`configurable["model_id"]`). The chain is invoked asynchronously using `chain.ainvoke(payload)`.

### Response Processing and Insight Persistence

The LLM's raw `response.content` undergoes two sanitization steps before storage:

- **`extract_text_content`** strips the response to plain text, removing any metadata wrappers.
- **`clean_thinking_content`** removes chain-of-thought markers or "thinking" meta-information that models like Claude or DeepSeek may emit.

If a `Source` instance is present in the state, the cleaned result is persisted as an insight using `await source.add_insight(transformation.title, cleaned_content)`, linking the transformation output directly to the source document.

## Graph Structure and API Integration

The transformation graph uses minimal wiring: a single node named `"agent"` connected from `START` to `END`. The compiled graph (`graph = agent_state.compile()`) is invoked from the API endpoint `POST /transformations/execute` in [`api/routers/transformations.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/transformations.py).

The endpoint accepts the `input_text`, selected `Transformation` record, and `model_id`, then returns the transformed output after the graph completes execution.

## Practical Implementation Examples

### Direct Graph Invocation (Python)

Invoke the transformation pipeline programmatically by compiling the graph with your source and model configuration:

```python
from open_notebook.graphs.transformation import graph as transformation_graph
from open_notebook.domain.transformation import Transformation
from open_notebook.domain.notebook import Source

# Assume a Source object already exists

source = await Source.get(source_id)

# Load a transformation definition (e.g., "Summarize")

transformation = await Transformation.get(transformation_id)

# Run the graph, providing the source and choosing a model

result = await transformation_graph.ainvoke(
    {
        "source": source,          # The Source object (optional if input_text supplied)

        "transformation": transformation,
    },
    config={"configurable": {"model_id": "openai:gpt-4o"}},  # model selection

)

print("Transformed output:", result["output"])

```

### Public API Execution (cURL)

Trigger transformations via the REST API when operating the Open-Notebook server:

```bash
curl -X POST https://localhost:5055/transformations/execute \
  -H "Content-Type: application/json" \
  -d '{
        "transformation_id": "summarize",
        "model_id": "anthropic:claude-3.5-sonnet",
        "input_text": "Long article text ..."
      }'

```

Response:

```json
{
  "output": "A concise summary of the article …",
  "translation_id": "summarize",
  "model_id": "anthropic:claude-3.5-sonnet"
}

```

### Customizing Default Instructions (cURL)

Modify the system-wide default prompts that prefix all transformation templates:

```bash
curl -X PUT https://localhost:5055/transformations/default-prompt \
  -H "Content-Type: application/json" \
  -d '{ "transformation_instructions": "Always keep the original tone." }'

```

## Key Implementation Files

| Component | File Path | Role |
|-----------|-----------|------|
| **Transformation Graph** | [`open_notebook/graphs/transformation.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/transformation.py) | Defines `TransformationState`, `run_transformation` node, and graph wiring. |
| **Transformation Model** | [`open_notebook/domain/transformation.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/transformation.py) | Pydantic schemas for `Transformation` records and `DefaultPrompts` configuration. |
| **Source Model** | [`open_notebook/domain/notebook.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/notebook.py) | Provides `full_text` property and `add_insight` method for persistence. |
| **LLM Provisioning** | [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) | Implements `provision_langchain_model` for model-agnostic chain creation. |
| **API Router** | [`api/routers/transformations.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/transformations.py) | Exposes `POST /transformations/execute` endpoint. |
| **Text Utilities** | [`open_notebook/utils/text_utils.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/text_utils.py) (extract), [`open_notebook/utils/__init__.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/__init__.py) (cleaning) | Utilities for normalizing LLM output before persistence. |

## Summary

- **LangGraph transformations** in Open-Notebook follow a single-node workflow defined in [`open_notebook/graphs/transformation.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/transformation.py).
- The system automatically falls back to `source.full_text` when `input_text` is not provided, enabling both ad-hoc and persisted content processing.
- Prompts combine `DefaultPrompts` (global instructions) with transformation-specific templates before being sent to provisioned models via `provision_langchain_model`.
- LLM responses are sanitized using `extract_text_content` and `clean_thinking_content` to remove meta-information and formatting artifacts.
- Results are persisted as source insights using `source.add_insight`, creating a permanent link between the transformation output and the original document.

## Frequently Asked Questions

### What happens if I don't provide input_text to a transformation?

The `run_transformation` node checks for `input_text` in the state and automatically falls back to `source.full_text` when the direct input is missing. This ensures the workflow processes the full extracted text of any persisted `Source` object, making the transformation flexible for both temporary text and stored documents.

### How does the system handle different LLM providers?

The `provision_langchain_model` function in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) dynamically constructs a LangChain chain based on the `model_id` passed in the LangGraph configuration (`configurable["model_id"]`). This allows the same graph to execute using OpenAI, Anthropic, or other compatible providers without code changes, requiring only the model identifier in the format `"provider:model-name"`.

### Where are transformation results stored?

When a `Source` object is present in the graph state, the cleaned transformation output is persisted via `await source.add_insight(transformation.title, cleaned_content)`. This attaches the result as an insight to the source document, making it available for future reference within the notebook interface. If no source is provided, the result is returned directly in the graph output without persistence.

### Can I customize the system prompts for all transformations?

Yes. The `DefaultPrompts` class in [`open_notebook/domain/transformation.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/transformation.py) stores global instructions that are optionally prefixed to every transformation-specific prompt. You can update these defaults via the `PUT /transformations/default-prompt` API endpoint, allowing you to inject persistent instructions—such as tone guidelines or formatting requirements—across all transformation executions.