How Open Notebook Graph Nodes Clean Thinking Content from AI Responses

Open Notebook strips internal reasoning from extended-thinking LLM outputs using regex patterns in text_utils.py, returning only the cleaned content through the clean_thinking_content function while preserving the raw thinking blocks for debugging.

Extended-thinking models such as Nemotron and Claude-Next embed their chain-of-thought reasoning inside `` blocks with proper opening and closing tags

• THINK_PATTERN_NO_OPEN: Catches malformed output where the opening tag is missing or incomplete

This dual-pattern approach ensures robust handling of streaming or truncated model outputs that might otherwise leak partial reasoning into the cleaned content.

parse_thinking_content Function

The parse_thinking_content function serves as the low-level extractor. It scans the raw AI response and returns a tuple containing (thinking_content, cleaned_content), separating the internal monologue from the user-facing output.

clean_thinking_content Convenience Wrapper

Lines 100–119 of text_utils.py define the clean_thinking_content function, which wraps the parser and discards the first tuple element. This helper returns only the cleaned string, making it ideal for graph nodes that need to forward content without the reasoning noise.

# open_notebook/utils/text_utils.py (conceptual)

def clean_thinking_content(text: str) -> str:
    _, cleaned = parse_thinking_content(text)
    return cleaned

Graph Node Implementation

Open Notebook integrates thinking-content cleaning at multiple points in its processing graphs to prevent malformed JSON parsing and ensure clean message storage.

Ask Graph Processing

In open_notebook/graphs/ask.py (lines 68–71 and 118–119), the system extracts and cleans content immediately after receiving the model response:

# open_notebook/graphs/ask.py

message_content = extract_text_content(ai_message.content)
cleaned_content = clean_thinking_content(message_content)   # ← removes thinking blocks

strategy = parser.parse(cleaned_content)                    # parse the JSON payload

Here, clean_thinking_content executes before the JSON parser attempts to parse the strategy, preventing syntax errors caused by stray `Actual JSON response" clean_content = clean_thinking_content(raw_content)

## Summary

- **[`open_notebook/utils/text_utils.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/text_utils.py)** defines the core regex patterns and the `clean_thinking_content` function that strips `` blocks with proper opening and closing tags, and **THINK_PATTERN_NO_OPEN** for malformed output where the opening tag is missing. This dual-pattern approach handles both complete and truncated model responses.

### How does clean_thinking_content differ from parse_thinking_content?

**`parse_thinking_content`** returns a tuple containing both the extracted thinking content and the cleaned text `(thinking_content, cleaned_content)`, which is useful when you need to log or analyze the model's reasoning. **`clean_thinking_content`** is a convenience wrapper that calls the parser and returns only the cleaned string, discarding the thinking portion entirely.

### Which graph nodes implement thinking content cleaning?

The cleaning logic appears in four primary graph nodes: **[`open_notebook/graphs/ask.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/ask.py)** (before JSON parsing), **[`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py)** (before updating conversation state), **[`open_notebook/graphs/source_chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/source_chat.py)** (before persisting LLM insights), and **[`open_notebook/graphs/transformation.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/transformation.py)** (before saving transformed data). Each node calls `clean_thinking_content` immediately after `extract_text_content`.

### Why is it necessary to strip thinking blocks from AI responses?

Extended-thinking models embed their internal reasoning inside `<think>` tags that can break JSON parsers or clutter user-facing outputs. Stripping these blocks ensures that downstream nodes receive clean, parseable content while preserving the ability to audit the model's reasoning through separate logging channels.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}