Open Notebook Custom Transformations Examples: Building Reusable AI Analysis Workflows

Open Notebook enables reusable AI analysis workflows through custom transformations—prompt templates that execute against sources via a LangGraph pipeline and store results as structured notes in SurrealDB.

Open Notebook provides a powerful extension mechanism for automating repetitive analysis tasks through custom transformations. These reusable prompt templates allow you to standardize how AI processes your research sources, from academic papers to web articles. This guide demonstrates practical open-notebook custom transformations examples using the actual source code from the lfnovo/open-notebook repository.

Architecture of Custom Transformations

Domain Model and Data Storage

The Transformation class in open_notebook/domain/transformation.py defines the core data structure for reusable analysis templates. Each transformation stores a name, title, description, prompt template, and default flag in SurrealDB. This domain model ensures type safety through Pydantic validation while maintaining flexibility for user-defined prompt engineering.

Execution Pipeline with LangGraph

The transformation engine lives in open_notebook/graphs/transformation.py. The run_transformation function and compiled graph handle the async execution flow, enabling parallel processing across multiple sources. Because the graph is built on LangGraph (StateGraph), transformations run asynchronously and can be distributed across large document collections without blocking the main application thread.

API Service Layer

The TransformationsService class in api/transformations_service.py exposes CRUD endpoints for managing transformation definitions and an execution endpoint for running transformations against arbitrary text. This service layer abstracts the database operations and graph execution, providing a clean interface for both programmatic and UI-driven interactions.

Frontend Integration

The React frontend consumes these APIs through frontend/src/lib/api/transformations.ts and manages component state via frontend/src/lib/hooks/use-transformations.ts. The hooks handle polling for background job completion and automatically refresh the notes list when transformations finish executing.

The Transformation Execution Flow

When you trigger a transformation, Open Notebook executes four distinct phases as implemented in lfnovo/open-notebook:

1. Input Selection – The system accepts either a single source or a batch of sources passed to the backend.
2. Prompt Assembly – The stored template combines with source content and global instructions from DefaultPrompts, then routes through provision_langchain_model to the selected LLM.
3. Response Post-Processing – Raw LLM output passes through clean_thinking_content to strip "thinking" artifacts and internal monologue before storage.
4. Result Delivery – The system creates a new note linked to the source, tagged with the transformation name, and displays it in the frontend.

Open Notebook Custom Transformations Examples

Creating a Domain-Specific Transformation via Python API

Use the TransformationsService to programmatically define new analysis templates without touching the database directly:

from api.transformations_service import transformations_service

# Create a new transformation that extracts a research paper’s key sections

new_transformation = transformations_service.create_transformation(
    name="academic-paper-analysis",
    title="Academic Paper Analysis",
    description="Extract research question, hypothesis, methodology, findings, etc.",
    prompt="""
    Analyze this academic paper and extract:
    1. **Research Question**: What problem does this address?
    2. **Hypothesis**: What did the authors predict?
    3. **Methodology**: How was the study conducted?
    4. **Key Findings**: What did they discover? (numbered list)
    5. **Limitations**: What caveats do the authors mention?
    6. **Future Work**: What do they suggest next?
    
    Be specific and cite page numbers where possible.
    """,
    apply_default=False,
)

print(f"Created transformation with ID: {new_transformation.id}")

The create_transformation method builds a Transformation object defined in open_notebook/domain/transformation.py, sends it to the API, and stores it in SurrealDB.

Executing a Transformation on Raw Text

Process arbitrary content without first ingesting it as a permanent source using the execution service:

source_text = """[Full PDF text of a research paper]"""

result = transformations_service.execute_transformation(
    transformation_id=new_transformation.id,
    input_text=source_text,
    model_id="gpt-4o-mini",   # any model supported by Esperanto

)

print("Transformation output:")
print(result["output"])

This call triggers the LangGraph workflow defined in open_notebook/graphs/transformation.py, which handles the LLM interaction and post-processing before returning the cleaned result.

Batch Processing Sources with React Hooks

Apply transformations to multiple sources concurrently using the frontend API client and React hooks:

// src/lib/hooks/use-transformations.ts (excerpt)
const applyTransformation = async (
  transformationId: string,
  sourceIds: string[],
  modelId: string,
) => {
  await Promise.all(
    sourceIds.map((srcId) =>
      api.executeTransformation({
        transformationId,
        inputText: getSourceContent(srcId), // pulls raw source text
        modelId,
      })
    )
  );
  // After all jobs finish the UI refreshes the notes list
};

The hook calls the /transformations/execute endpoint for each source ID, enabling parallel execution across your document collection.

Creating Transformations Through the Web Interface

Follow this workflow to create custom transformations using the UI components defined in docs/3-USER-GUIDE/transformations.md:

1. Navigate to the Transformations page and click Create New.
2. Fill the configuration form with name, title, description, and the prompt template—the UI sends a POST request to api/transformations/create.
3. Save and activate—the new template appears in the transformation list and can be selected for batch runs against selected sources.

Key Implementation Files

File	Purpose
open_notebook/domain/transformation.py	Pydantic model defining transformation records stored in SurrealDB
open_notebook/graphs/transformation.py	LangGraph state machine executing transformations against sources
api/transformations_service.py	Service layer exposing CRUD and execution endpoints
frontend/src/lib/api/transformations.ts	TypeScript API client for transformation endpoints
frontend/src/lib/hooks/use-transformations.ts	React hook for UI components to trigger transformations
docs/3-USER-GUIDE/transformations.md	User documentation for creating custom templates

Summary

• Open Notebook custom transformations are reusable prompt templates stored as domain objects in SurrealDB via open_notebook/domain/transformation.py.
• The LangGraph-based execution pipeline in open_notebook/graphs/transformation.py enables async, parallel processing of multiple sources through run_transformation.
• The TransformationsService API layer in api/transformations_service.py provides both CRUD operations and direct execution capabilities.
• Frontend integration uses TypeScript API clients and React hooks to manage transformation state and batch operations across source collections.
• Results are automatically cleaned using clean_thinking_content and stored as structured notes linked to their source documents.

Frequently Asked Questions

What is the difference between default and custom transformations in Open Notebook?

Default transformations ship with the application and apply to all sources automatically when the apply_default flag is set to True, while custom transformations are user-defined templates that target specific analysis workflows. The Transformation class in open_notebook/domain/transformation.py tracks this distinction via the boolean apply_default field, allowing you to designate templates that run automatically versus those requiring manual selection.

How does Open Notebook handle LLM response cleaning during transformation execution?

The execution graph in open_notebook/graphs/transformation.py includes a clean_thinking_content function that strips "thinking" artifacts and internal monologue from raw LLM outputs before persisting them as notes. This ensures clean, readable results without model-specific formatting noise, regardless of which provider you configure through provision_langchain_model.

Can I run custom transformations on multiple sources simultaneously?

Yes. The LangGraph architecture supports parallel execution across source batches. The use-transformations.ts React hook demonstrates this pattern by wrapping multiple API calls in Promise.all, allowing concurrent execution against the api/transformations_service.py endpoint without blocking the UI.

Where are transformation definitions stored in the Open Notebook architecture?

Transformation definitions persist in SurrealDB as records managed by the domain model in open_notebook/domain/transformation.py. The TransformationsService abstracts database operations, while the frontend accesses these definitions through the TypeScript client in frontend/src/lib/api/transformations.ts, ensuring type safety across the stack.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}