# How to Use the betterhtmlchunking tree_representation API to Retrieve Node Metadata (text_length and html_length) > Learn to use the betterhtmlchunking tree_representation API to access node metadata like text_length and html_length via the xpaths_metadata dictionary. Unlock valuable DOM information. - Repository: [Carlos A. Planchón/betterhtmlchunking](https://github.com/carlosplanchon/betterhtmlchunking) - Tags: how-to-guide - Published: 2026-02-26 --- **The `DOMTreeRepresentation` class exposes node metadata—including `text_length` and `html_length`—through the `xpaths_metadata` dictionary, which maps positional XPaths to `NodeMetadata` instances.** The `betterhtmlchunking` library provides a robust `tree_representation` API that parses HTML into a navigable DOM tree and computes per-node metadata. This metadata includes the character count of visible text and the serialized HTML length, enabling precise content analysis and chunking strategies. ## Understanding the DOMTreeRepresentation Class The `DOMTreeRepresentation` class, defined in [`betterhtmlchunking/tree_representation.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_representation.py), serves as the primary interface for DOM analysis. When initialized with an HTML string, it performs three core operations: 1. **Parses the HTML** using BeautifulSoup to create a navigable soup object. 2. **Computes positional XPaths** for every element via the `compute_xpaths_data` method. 3. **Builds a tree representation** using `treelib` and attaches metadata to each node. This process populates the `xpaths_metadata` attribute, which stores `NodeMetadata` objects keyed by their positional XPath strings. ## Accessing Node Metadata via xpaths_metadata ### The NodeMetadata Data Structure Each entry in `xpaths_metadata` is an instance of the `NodeMetadata` class (lines 58-80 in [`betterhtmlchunking/tree_representation.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_representation.py)). This dataclass contains: - **`text_length`**: The character count of the node's rendered text content (excluding HTML tags). - **`html_length`**: The character count of the node's serialized HTML representation. These metrics allow you to evaluate content density and make informed decisions about chunking boundaries. ### Retrieving Metadata by XPath To access metadata for a specific node, use its positional XPath as a key in the `xpaths_metadata` dictionary: ```python from betterhtmlchunking.tree_representation import DOMTreeRepresentation html_content = """

Hello World!

Second paragraph with more text content.

""" # Initialize the representation rep = DOMTreeRepresentation(website_code=html_content) # Access metadata using the positional XPath xpath = "/html/body/div[1]/p[2]" metadata = rep.xpaths_metadata[xpath] print(f"Text length: {metadata.text_length}") print(f"HTML length: {metadata.html_length}") ``` ## Alternative Access Methods ### Via the treelib Tree Node The `DOMTreeRepresentation` exposes the underlying `treelib.Tree` instance through the `tree` attribute. Each node in this tree stores its `NodeMetadata` in the `data` attribute, providing an alternative access path: ```python # Get the tree node directly tree_node = rep.tree.get_node("/html/body/div[1]/p[1]") node_metadata = tree_node.data # Same object as rep.xpaths_metadata[xpath] assert node_metadata is rep.xpaths_metadata["/html/body/div[1]/p[1]"] ``` This approach is useful when traversing the tree structure using `treelib` methods like `children()` or `subtree()`. ## Practical Code Example The following complete example demonstrates initializing the representation, listing available XPaths, and retrieving specific node metadata: ```python from betterhtmlchunking.tree_representation import DOMTreeRepresentation html_source = """ Demo Page

Main Title

This is the first paragraph with some text.

This is the second paragraph with even more text content to analyze.

""" # Step 1: Create the representation rep = DOMTreeRepresentation(website_code=html_source) # Step 2: View all available positional XPaths print("Available XPaths:") for xpath in rep.pos_sorted_xpaths: print(f" {xpath}") # Step 3: Retrieve metadata for specific nodes target_xpath = "/html/body/article[1]/p[1]" if target_xpath in rep.xpaths_metadata: meta = rep.xpaths_metadata[target_xpath] print(f"\nNode: {target_xpath}") print(f" Text length: {meta.text_length}") print(f" HTML length: {meta.html_length}") # Step 4: Alternative access via tree node tree_node = rep.tree.get_node(target_xpath) print(f"\nVia tree node data attribute:") print(f" Text length: {tree_node.data.text_length}") ``` ## Summary - The `DOMTreeRepresentation` class in [`betterhtmlchunking/tree_representation.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_representation.py) parses HTML and computes per-node metadata automatically. - **Node metadata**—including `text_length` and `html_length`—is stored in `NodeMetadata` objects accessible via the `xpaths_metadata` dictionary. - Use **positional XPaths** (e.g., `/html/body/div[1]/p[2]`) as keys to retrieve specific node metadata. - Access metadata alternatively through the `data` attribute of nodes in the underlying `treelib.Tree` instance. ## Frequently Asked Questions ### What is the difference between text_length and html_length in betterhtmlchunking? The `text_length` attribute represents the character count of the node's visible text content after stripping all HTML tags, while `html_length` represents the character count of the node's serialized HTML representation including tags and attributes. For example, a paragraph containing `

Hello World

` would have a `text_length` of 11 (for "Hello World") but a larger `html_length` accounting for the markup. ### How does DOMTreeRepresentation generate positional XPaths? The `compute_xpaths_data` method in [`betterhtmlchunking/tree_representation.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_representation.py) traverses the BeautifulSoup parse tree and generates positional XPaths (POS-XPaths) by tracking the index of each element among its siblings with the same tag name. These XPaths follow the format `/html/body/div[1]/p[2]`, where the indices indicate the position within the parent container, ensuring every node has a unique, deterministic identifier. ### Can I modify node metadata after the tree is created? The `NodeMetadata` objects stored in `xpaths_metadata` are standard Python dataclass instances, so you can technically modify their attributes (such as `text_length` or `html_length`) after retrieval. However, these modifications only affect the in-memory object and do not trigger recalculation of the DOM structure. To update the representation after structural changes, you must create a new `DOMTreeRepresentation` instance with the modified HTML. ### Where is the NodeMetadata class defined in the source code? The `NodeMetadata` class is defined in [`betterhtmlchunking/tree_representation.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_representation.py) at approximately lines 58-80. This dataclass encapsulates the metadata for individual DOM nodes, including the node index, `text_length`, and `html_length` attributes. The class is imported and used throughout the module to populate the `xpaths_metadata` dictionary during the tree construction process.