# How the ROIMaker Class Handles Nodes That Exceed the max_node_repr_length Limit

> Learn how the ROIMaker class isolates and processes nodes exceeding max_node_repr_length. Discover efficient handling of oversized nodes in betterhtmlchunking to optimize performance.

- Repository: [Carlos A. Planchón/betterhtmlchunking](https://github.com/carlosplanchon/betterhtmlchunking)
- Tags: internals
- Published: 2026-02-26

---

**The ROIMaker class immediately isolates any child node whose representation length meets or exceeds the configured `max_node_repr_length`, sealing the current region of interest and enqueueing the oversized node for deeper recursive processing.**

The `ROIMaker` class in the `carlosplanchon/betterhtmlchunking` repository implements a critical safety mechanism to ensure that generated HTML chunks never exceed specified size limits. When processing a DOM node's children, the class must handle nodes that individually exceed the `max_node_repr_length` threshold without breaking the chunking algorithm's integrity.

## The Oversized Node Detection Algorithm

The detection logic resides in [`betterhtmlchunking/tree_regions_system.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_regions_system.py) and follows a strict rule-based approach to identify nodes that cannot fit within the current region.

### Measuring Node Length with get_node_repr_length

Before processing, `ROIMaker` determines each child's size using `get_node_repr_length` (lines 94-101). This method returns either the text or HTML length depending on the `repr_length_compared_by` setting:

```python
def get_node_repr_length(self, xpath: str) -> int:
    if self.repr_length_compared_by == ReprLengthComparisionBy.TEXT_LENGTH:
        return self.tree_representation.get_node_text_length(xpath=xpath)
    else:
        return self.tree_representation.get_node_html_length(xpath=xpath)

```

### The Iteration Logic in _process_children

The core iteration occurs in `_process_children` (lines 102-110), which walks through `self.children_tags`—the XPaths of the node's direct children. For each child, the algorithm evaluates whether the current node can be added to the growing region of interest or if it triggers a boundary condition.

### The Critical Threshold Check

The decisive logic for handling oversized nodes appears at lines 113-121. When a child's length meets or exceeds `max_node_repr_length`, the algorithm immediately triggers isolation:

```python
if node_length >= self.max_node_repr_length:
    if current_region.pos_xpath_list:
        self.regions_of_interest_list.append(current_region)
        current_region = RegionOfInterest()
    self.children_to_enqueue.append(child_xpath)
    continue

```

This check ensures that no single node exceeding the limit ever enters a region of interest.

## Isolating and Enqueueing Oversized Children

When the threshold check triggers, `ROIMaker` executes a three-step isolation protocol to maintain chunk size integrity.

First, **seal the current region**. If `current_region.pos_xpath_list` contains any accumulated children, the region is finalized and appended to `self.regions_of_interest_list`. This prevents partial regions from remaining open when an oversized sibling interrupts the sequence.

Second, **enqueue for recursion**. The oversized child's XPath is appended to `self.children_to_enqueue`. According to the source code in [`betterhtmlchunking/tree_regions_system.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_regions_system.py), the `TreeRegionsSystem` driver later dequeues these XPaths and instantiates a new `ROIMaker` for each, effectively descending one level deeper into the DOM tree.

Third, **continue iteration**. The algorithm proceeds to the next sibling without adding the oversized node to the current region, maintaining the invariant that all regions contain only nodes smaller than the limit.

## Practical Implementation Examples

The following examples demonstrate how the oversized node handling operates in practice using the `betterhtmlchunking` library.

### Using TreeRegionsSystem (High-Level API)

For most use cases, interact with the chunking system through `TreeRegionsSystem`, which internally manages `ROIMaker` instances and oversized node recursion:

```python
from betterhtmlchunking.tree_representation import DOMTreeRepresentation
from betterhtmlchunking.tree_regions_system import TreeRegionsSystem, ReprLengthComparisionBy

html = "<div><p>Short text</p>" + "<p>" + "x" * 15000 + "</p></div>"
tree_repr = DOMTreeRepresentation(html)

system = TreeRegionsSystem(
    tree_representation=tree_repr,
    max_node_repr_length=10_000,
    repr_length_compared_by=ReprLengthComparisionBy.HTML_LENGTH,
)

chunks = system.sorted_roi_by_pos_xpath
for idx, roi in chunks.items():
    print(f"Chunk {idx}: {len(roi.pos_xpath_list)} nodes, {roi.repr_length} chars")

```

In this example, the second `<p>` element exceeds 10,000 characters, so `ROIMaker` isolates it into `children_to_enqueue` for separate processing rather than including it in the initial chunk.

### Direct ROIMaker Instantiation (Unit Testing)

For testing or custom chunking logic, instantiate `ROIMaker` directly to observe the oversized node handling:

```python
from betterhtmlchunking.tree_regions_system import ROIMaker, ReprLengthComparisionBy
from betterhtmlchunking.tree_representation import DOMTreeRepresentation

html = "<section><h1>Title</h1><article>" + "Content" * 2000 + "</article></section>"
tree_repr = DOMTreeRepresentation(html)

roi_maker = ROIMaker(
    node_xpath="/html/body/section",
    children_tags=["/html/body/section/h1", "/html/body/section/article"],
    tree_representation=tree_repr,
    max_node_repr_length=1_000,
    repr_length_compared_by=ReprLengthComparisionBy.HTML_LENGTH,
)

print("Regions of Interest:", len(roi_maker.regions_of_interest_list))
print("Oversized nodes to enqueue:", roi_maker.children_to_enqueue)

```

If the `<article>` element's HTML length exceeds 5,000 characters, it appears in `children_to_enqueue` while the `<h1>` element is safely contained within `regions_of_interest_list`.

## Summary

- **Immediate isolation**: When `ROIMaker` encounters a child node with `node_length >= max_node_repr_length`, it immediately seals the current region and excludes the oversized node.
- **Recursive enqueueing**: Oversized nodes are stored in `children_to_enqueue` for the `TreeRegionsSystem` driver to process in subsequent, deeper recursion levels.
- **Invariant preservation**: The algorithm guarantees that no region in `regions_of_interest_list` contains a node exceeding the configured limit, maintaining the integrity of the HTML chunking process.
- **Configurable measurement**: Node length is determined by `get_node_repr_length`, which supports both text and HTML length comparisons via the `repr_length_compared_by` parameter.

## Frequently Asked Questions

### What happens if a node exactly equals max_node_repr_length?

If a node's length is exactly equal to `max_node_repr_length`, the condition `node_length >= self.max_node_repr_length` evaluates to true, triggering the same isolation logic as oversized nodes. The node is enqueued for deeper processing rather than being added to the current region, ensuring strict adherence to the size boundary.

### How does ROIMaker prevent oversized nodes from appearing in final regions?

`ROIMaker` implements a defensive check in the `_process_children` method (lines 113-121 of [`betterhtmlchunking/tree_regions_system.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_regions_system.py)). Before adding any child to the current `RegionOfInterest`, it verifies the child's length against the threshold. If the check fails, the current region is immediately sealed and appended to `regions_of_interest_list`, while the oversized child is routed to `children_to_enqueue`, completely bypassing the region collection.

### Can I adjust max_node_repr_length after creating a ROIMaker instance?

No, `max_node_repr_length` is set during instantiation via the constructor and stored as an instance attribute (`self.max_node_repr_length`). The class does not provide setter methods or property decorators to modify this value after construction. To use a different limit, you must create a new `ROIMaker` instance with the desired parameter.

### Where does the recursive processing of oversized nodes occur?

The recursive processing is orchestrated by the `TreeRegionsSystem` class, which acts as the driver for `ROIMaker`. After `ROIMaker` populates `children_to_enqueue` with oversized node XPaths, the `TreeRegionsSystem` dequeues these paths and instantiates new `ROIMaker` instances for each one, effectively descending one level deeper into the DOM tree. This driver logic is implemented in the same [`betterhtmlchunking/tree_regions_system.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_regions_system.py) file that contains the `ROIMaker` class.