# Create Custom Content Transformation Processing Sources with AI in Open Notebook

> Learn to create custom AI-driven content transformations in Open Notebook. Leverage SurrealDB, LangGraph, and LLMs for powerful processing pipelines.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-07-05

---

**Open Notebook enables users to create reusable AI-driven transformations that process any source content through a declarative pipeline combining SurrealDB storage, LangGraph execution, and multi-provider LLM support.**

You can create custom content transformation processing sources with AI by defining reusable prompt templates that automatically process PDFs, webpages, and audio files through a structured LangGraph pipeline. The transformation system in Open Notebook treats each custom prompt as a first-class entity stored in SurrealDB, allowing you to apply consistent AI processing across diverse content types. This architecture separates the transformation definition from execution, enabling non-technical users to craft prompts while the backend handles complex model provisioning and graph orchestration.

## Understanding the Transformation Architecture

A **transformation** in Open Notebook is fundamentally a stored AI prompt combined with execution logic. The system persists transformation definitions as records containing a name, title, description, and the LLM prompt template. When executed, the transformation processes source content through a dedicated LangGraph that handles model selection, prompt composition, and result storage.

The architecture relies on four core components working in sequence:

- **Domain Model** ([`open_notebook/domain/transformation.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/transformation.py)) - Defines the `Transformation` class and `DefaultPrompts` singleton for global instruction management
- **API Router** ([`api/routers/transformations.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/transformations.py)) - Exposes HTTP endpoints for CRUD operations and execution via FastAPI
- **Graph Execution** ([`open_notebook/graphs/transformation.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/transformation.py)) - Implements the LangGraph that orchestrates the LLM interaction
- **Service Layer** ([`api/transformations_service.py`](https://github.com/lfnovo/open-notebook/blob/main/api/transformations_service.py)) - Provides a Pythonic wrapper around the HTTP client for programmatic access

## Define a Custom Transformation

Transformations are stored as database records in SurrealDB through the `Transformation` domain model. Each record encapsulates the prompt template and metadata required for reusable AI processing.

The `Transformation` class in [`open_notebook/domain/transformation.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/transformation.py) provides the persistence layer, while the API router in [`api/routers/transformations.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/transformations.py) handles HTTP requests to create these records. When you create a transformation, you specify the prompt that will be sent to the LLM along with optional model overrides.

### Persisting Transformation Definitions

The backend exposes REST endpoints for transformation management. The `Transformation.save()` method persists the record to SurrealDB, making the transformation available for execution across the application. The service layer wraps these operations for convenient Python usage.

## Execute AI Transformations with LangGraph

Execution occurs through a dedicated LangGraph defined in [`open_notebook/graphs/transformation.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/transformation.py). This graph contains a single node `run_transformation` that orchestrates the entire AI processing pipeline.

### The Transformation Graph Flow

The execution flow follows a precise sequence:

1. **Prompt Composition** - The system merges the stored transformation prompt with optional default instructions from the `DefaultPrompts` singleton
2. **Model Provisioning** - `provision_langchain_model` (from [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py)) instantiates the correct LLM using the Esperanto multi-provider manager
3. **Content Processing** - The graph sends the composed prompt plus source content to the provisioned model
4. **Output Cleaning** - `clean_thinking_content` removes any "thinking" tokens or extraneous formatting from the LLM response
5. **Result Storage** - The cleaned output is stored as an insight on the original source, making it searchable alongside the raw content

The graph is invoked asynchronously using `transformation_graph.ainvoke`, ensuring non-blocking execution for long-running AI operations.

## Programmatic Usage Examples

You can interact with the transformation system through multiple interfaces: the Python service layer, direct API calls, or the TypeScript frontend client.

### Creating Transformations via the Service Layer

Use `TransformationsService` to create reusable transformation templates programmatically:

```python
from api.transformations_service import transformations_service

new_trans = transformations_service.create_transformation(
    name="summarize",
    title="Summarize Content",
    description="Generates a concise summary of the source text.",
    prompt="You are a helpful assistant. Summarize the following content in 3 sentences:",
    apply_default=True,
)
print(new_trans.id, new_trans.title)

```

This service layer method corresponds to the `POST /transformations` endpoint in [`api/routers/transformations.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/transformations.py).

### Running Transformations on Source Content

Execute a stored transformation against any text content using the service layer:

```python
result = transformations_service.execute_transformation(
    transformation_id="transform-123",
    input_text="Full text extracted from a PDF …",
    model_id=None,  # uses the default model for the transformation

)
print(result["output"])

```

Under the hood, this triggers `transformation_graph.ainvoke` with the transformation ID and input text, routing through the LangGraph defined in [`open_notebook/graphs/transformation.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/transformation.py).

### Managing Global Default Prompts

The `DefaultPrompts` singleton stores global instructions that automatically prefix every transformation unless explicitly disabled. Update these defaults through the API client:

```python
from api.client import api_client

api_client.put_default_prompt({
    "transformation_instructions": "Always keep the tone formal and avoid bullet points."
})

```

This corresponds to the `PUT /transformations/default-prompt` endpoint, which modifies the base instructions applied to all transformations.

## Frontend Integration

The TypeScript API wrapper in [`frontend/src/lib/api/transformations.ts`](https://github.com/lfnovo/open-notebook/blob/main/frontend/src/lib/api/transformations.ts) mirrors the backend routes, enabling React components to manage transformations directly:

```typescript
import { transformationsApi } from '@/lib/api/transformations';

// List all transformations
const list = await transformationsApi.list();

// Execute one
const execRes = await transformationsApi.execute({
  transformation_id: list[0].id,
  input_text: source.fullText,
});
console.log(execRes.output);

```

This frontend integration maps directly to the FastAPI routes, ensuring type safety and consistent behavior across the stack.

## Summary

- **Transformations are declarative AI prompts** stored as SurrealDB records via the `Transformation` class in [`open_notebook/domain/transformation.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/transformation.py)
- **Execution flows through LangGraph** in [`open_notebook/graphs/transformation.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/transformation.py), specifically the `run_transformation` node that handles prompt composition and model provisioning
- **Multi-provider support** comes from the Esperanto model manager in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py), enabling flexible LLM selection per transformation
- **Global defaults** can be configured via the `DefaultPrompts` singleton to ensure consistent AI behavior across all transformations
- **Full-stack accessibility** is provided through the Python service layer ([`api/transformations_service.py`](https://github.com/lfnovo/open-notebook/blob/main/api/transformations_service.py)) and TypeScript frontend client ([`frontend/src/lib/api/transformations.ts`](https://github.com/lfnovo/open-notebook/blob/main/frontend/src/lib/api/transformations.ts))

## Frequently Asked Questions

### What is a transformation in Open Notebook?

A transformation is a reusable AI-driven prompt template that processes source content through a structured pipeline. According to the Open Notebook source code, it is defined as a `Transformation` record containing a name, title, description, and the LLM prompt. These records are stored in SurrealDB and can be applied to any source type including PDFs, webpages, and audio files.

### How does the default prompt system work?

The system uses a singleton `DefaultPrompts` record stored in [`open_notebook/domain/transformation.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/transformation.py) to maintain global instructions. When executing a transformation with `apply_default=True`, the graph automatically prefixes these global instructions to the specific transformation prompt. You can modify the default prompt via the `PUT /transformations/default-prompt` endpoint, affecting all subsequent transformations unless they explicitly override the behavior.

### Can I use different AI models for different transformations?

Yes, each transformation record supports an optional model override. When calling `execute_transformation`, you can specify a `model_id` parameter to use a specific LLM, or pass `None` to use the system default. The `provision_langchain_model` function in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py) handles the actual model instantiation through the Esperanto multi-provider manager, supporting various LLM backends.

### Where are transformation results stored?

After the LangGraph processes the content through `run_transformation` and cleans the output using `clean_thinking_content`, the results are stored as insights on the original source record in SurrealDB. This makes transformed content searchable alongside the original source material, maintaining the relationship between raw input and AI-processed output within the Open Notebook data model.