# What Happens When max_node_repr_length Is Larger Than the Document in BetterHTMLChunking

> Discover what happens when max_node_repr_length exceeds your document size in BetterHTMLChunking. Your entire document becomes a single ROI, preventing empty results.

- Repository: [Carlos A. Planchón/betterhtmlchunking](https://github.com/carlosplanchon/betterhtmlchunking)
- Tags: internals
- Published: 2026-02-26

---

**When the `max_node_repr_length` threshold exceeds the total representation length of the HTML document, the Tree Regions System automatically falls back to treating the entire document as a single Region of Interest (ROI) instead of returning an empty list.**

The `betterhtmlchunking` library by carlosplanchon partitions HTML documents into manageable chunks based on representation length limits. When you configure a `max_node_repr_length` value larger than the document itself, the algorithm cannot create smaller sub-regions because every node—including the root—fits comfortably within your specified threshold. In this edge case, the system gracefully degrades to a single-ROI strategy to ensure downstream processing always has valid content to work with.

## How the Tree Regions System Creates ROIs

The **Tree Regions System** processes the DOM tree using a breadth-first search (BFS) traversal to group child nodes into *regions of interest* (ROIs). Each ROI represents a cluster of nodes whose combined representation length does not exceed the user-supplied `max_node_repr_length` threshold.

During normal operation, the system evaluates node representation lengths—whether measured by HTML character count, text content, or token count—and partitions the document accordingly. However, when your threshold is set higher than the entire document's representation length, no valid subdivision points exist. The system detects this condition after sorting the collected ROI objects and finding an empty list despite the DOM containing valid nodes.

## Fallback Behavior When No ROI Is Found Due to max_node_repr_length Settings

When `max_node_repr_length` is configured larger than the document size, no ROIs are generated during the initial traversal. In this scenario, the library executes a specific fallback block inside [`betterhtmlchunking/tree_regions_system.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_regions_system.py), where the `TreeRegionsSystem.start` method detects the empty sorted list while the DOM contains at least one node, then constructs a single `RegionOfInterest` spanning the root node.

The fallback logic uses the first XPath in `pos_xpaths_list` (which corresponds to the document root) and creates a synthetic ROI covering the entire document:

```python

# betterhtmlchunking/tree_regions_system.py

# … after processing all nodes …

if sorted_regions == [] and len(self.tree_representation.pos_xpaths_list) > 0:
    logger.info(
        "No ROIs found with current settings, using entire document as single ROI"
    )
    node_xpath = self.tree_representation.pos_xpaths_list[0]

    node = self.tree_representation.tree.get_node(node_xpath)
    node_repr_length = self.get_node_repr_length(node)

    roi = RegionOfInterest()
    roi.pos_xpath_list = [node_xpath]
    roi.repr_length = node_repr_length
    roi.node_is_roi = True

    sorted_regions = [roi]

```

*Source:* [TreeRegionsSystem.start implementation](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_regions_system.py#L14-L22)

This implementation guarantees that **downstream rendering and chunking operations always receive at least one valid region**, preventing pipeline failures due to empty ROI lists.

## Code Examples: Triggering vs. Avoiding the Fallback

### Example 1: Triggering the Single-ROI Fallback

Setting `MAX_NODE_REPR_LENGTH` to a value far exceeding your document size forces the fallback mechanism. The system logs an informational message and returns the root node as the sole ROI:

```python
from betterhtmlchunking.main import DomRepresentation
from betterhtmlchunking.tree_regions_system import ReprLengthComparisionBy

# Small HTML document

html = "<html><body><p>Hello world</p></body></html>"

# Threshold far larger than document size

doc = DomRepresentation(
    MAX_NODE_REPR_LENGTH=10_000,      # exceeds total document length

    website_code=html,
    repr_length_compared_by=ReprLengthComparisionBy.HTML_LENGTH,
)

doc.start(maximal_verbose=True)

# Inspect the resulting ROI

roi = list(doc.tree_regions_system.sorted_roi_by_pos_xpath.values())[0]
print("ROI XPaths :", roi.pos_xpath_list)
print("ROI length :", roi.repr_length)
print("Is whole node ROI :", roi.node_is_roi)

```

**Output:**

```

ROI XPaths : ['/html']
ROI length : 54          # total HTML length of the document

Is whole node ROI : True

```

The log output confirms the fallback activation:

```

INFO  No ROIs found with current settings, using entire document as single ROI

```

### Example 2: Normal Multi-ROI Behavior

With a sensible `max_node_repr_length` value smaller than the document, the system creates multiple ROIs as expected:

```python
doc = DomRepresentation(
    MAX_NODE_REPR_LENGTH=30,   # smaller than total document size

    website_code=html,
    repr_length_compared_by=ReprLengthComparisionBy.HTML_LENGTH,
)

doc.start()
print(len(doc.tree_regions_system.sorted_roi_by_pos_xpath))

# Output: 2 (or more, depending on HTML structure)

```

Here the algorithm successfully subdivides the DOM because the limit forces the creation of smaller, distinct chunks.

## Summary

- **Automatic fallback**: When `max_node_repr_length` exceeds the document's total representation length, the system cannot create smaller ROIs and automatically treats the entire document as a single region.
- **Root node coverage**: The fallback ROI always uses the first XPath in `pos_xpaths_list`, which corresponds to the document root (`/html`), ensuring complete content coverage.
- **Zero pipeline failures**: The `TreeRegionsSystem.start` method in [`betterhtmlchunking/tree_regions_system.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_regions_system.py) guarantees at least one ROI exists before proceeding to rendering and chunking stages.
- **Observable behavior**: The fallback triggers an INFO-level log message ("No ROIs found with current settings...") and sets `node_is_roi=True` on the resulting region object.

## Frequently Asked Questions

### What is the max_node_repr_length parameter in BetterHTMLChunking?

The `max_node_repr_length` parameter defines the maximum allowed representation length for any single Region of Interest (ROI) during the HTML chunking process. According to the source code in [`betterhtmlchunking/main.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/main.py), this threshold controls how the `DomRepresentation` class partitions the DOM tree, with the comparison method determined by `repr_length_compared_by` (HTML length, text length, or token count).

### Why does the library return the entire document as a single ROI instead of an empty list?

The library implements a defensive programming pattern in `TreeRegionsSystem.start` to prevent downstream pipeline failures. When `sorted_regions` is empty but the DOM contains nodes, the code constructs a synthetic `RegionOfInterest` covering the root node. This ensures that rendering and chunking operations always have valid content to process, avoiding null pointer exceptions or empty iteration errors in consumer code.

### How can I detect when the fallback mechanism has been triggered?

You can detect fallback activation through two indicators: first, the system logs the message "No ROIs found with current settings, using entire document as single ROI" at INFO level when `maximal_verbose=True`. Second, inspect the resulting ROI object—if `node_is_roi` is `True` and the `pos_xpath_list` contains only the root XPath (`/html`), the fallback has been applied.

### What happens if I set max_node_repr_length to an extremely large value like 100,000?

Setting `max_node_repr_length` to 100,000 on a typical HTML document will almost certainly trigger the fallback mechanism described in [`betterhtmlchunking/tree_regions_system.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_regions_system.py). Since most web pages have representation lengths far below this threshold, the algorithm cannot identify valid subdivision points and returns the entire document as a single ROI with the root node's XPath, effectively disabling granular chunking for that document.