# How to Access Positional XPaths to Trace HTML Chunks Back to Source in betterhtmlchunking
> Discover how to access positional XPaths in betterhtmlchunking using RegionOfInterest.pos_xpath_list to precisely trace HTML chunks back to their original source locations.
- Repository: [Carlos A. Planchón/betterhtmlchunking](https://github.com/carlosplanchon/betterhtmlchunking)
- Tags: how-to-guide
- Published: 2026-02-26
---
**You can access positional XPaths through `RegionOfInterest.pos_xpath_list` after processing HTML with `DomRepresentation`, which maps each chunk to its exact source location using indexed paths like `/html/body/div[2]/p[1]`.**
The `betterhtmlchunking` library generates intelligent chunks from HTML documents while preserving traceability to the original markup. When you need to **access positional XPaths** to verify chunk origins or debug content extraction, the library exposes these paths through its DOM tree representation and region detection system.
## Understanding Positional XPaths in betterhtmlchunking
`betterhtmlchunking` builds a full **DOM-tree representation** of the input HTML. During this process, every element receives a **positional XPath**—a path containing the exact index of each sibling (e.g., `/html/body/div[2]/p[1]`).
The library stores these XPaths in two primary locations:
- **`DOMTreeRepresentation.pos_xpaths_list`** – A flat list of **all** positional XPaths generated from the document (e.g., `["/html", "/html/body", "/html/body/div[1]", …]`)
- **`RegionOfInterest.pos_xpath_list`** – The subset of XPaths that compose a **single chunk (ROI)** (e.g., `["/html/body/div[2]/h1", "/html/body/div[2]/p[1]"]`)
When the **TreeRegionsSystem** finishes processing, it creates a dictionary `TreeRegionsSystem.sorted_roi_by_pos_xpath` that maps a numeric chunk index to a `RegionOfInterest` instance. Each ROI contains the exact positional XPaths that generated that chunk.
## Accessing XPaths from the DOM Tree Representation
### Retrieving All XPaths from DOMTreeRepresentation
The `DOMTreeRepresentation` class in [`betterhtmlchunking/tree_representation.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_representation.py) computes positional XPaths during initialization. You can access the complete list via the `pos_xpaths_list` attribute:
```python
from betterhtmlchunking.main import DomRepresentation
dom = DomRepresentation(
MAX_NODE_REPR_LENGTH=2000,
website_code="",
)
dom.start()
# Access all positional XPaths in the document
all_xpaths = dom.tree_representation.pos_xpaths_list
print(all_xpaths)
# Output: ['/html', '/html/body', '/html/body/div', '/html/body/div/p']
```
### Mapping XPaths to BeautifulSoup Elements
To trace an XPath back to its original HTML element, use the `xpaths_metadata` dictionary. Each entry maps an XPath to a `NodeMetadata` object containing the `bs4_elem` (BeautifulSoup Tag):
```python
# Get the BeautifulSoup Tag for a specific XPath
xpath = "/html/body/div[1]/p[2]"
metadata = dom.tree_representation.xpaths_metadata[xpath]
original_tag = metadata.bs4_elem
print(original_tag.prettify())
```
This gives you the exact source snippet that contributed to the chunk.
## Tracing Chunks via RegionOfInterest
### Accessing the Sorted ROI Dictionary
The `TreeRegionsSystem` class in [`betterhtmlchunking/tree_regions_system.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_regions_system.py) groups DOM nodes into chunks (Regions of Interest). After processing, access `sorted_roi_by_pos_xpath` to map chunk indices to their source XPaths:
```python
# Access the dictionary mapping chunk index to RegionOfInterest
roi_dict = dom.tree_regions_system.sorted_roi_by_pos_xpath
# Get chunk 0's source XPaths
first_chunk_xpaths = roi_dict[0].pos_xpath_list
print(first_chunk_xpaths)
```
### Extracting XPaths for Specific Chunks
Iterate over the sorted dictionary to trace every chunk back to its origin:
```python
for idx, roi in dom.tree_regions_system.sorted_roi_by_pos_xpath.items():
print(f"--- Chunk {idx} ---")
for xpath in roi.pos_xpath_list:
print(xpath)
```
*Example output:*
```
--- Chunk 0 ---
/html/body/div[1]/h1
--- Chunk 1 ---
/html/body/div[1]/p[1]
/html/body/div[1]/p[2]
--- Chunk 2 ---
/html/body/div[2]/ul/li[1]
/html/body/div[2]/ul/li[2]
```
## Practical Code Examples for XPath Retrieval
### Function to Get Source XPaths by Chunk Index
```python
def get_chunk_source_xpaths(dom, chunk_idx: int) -> list[str]:
"""Return the list of positional XPaths that built the requested chunk."""
roi = dom.tree_regions_system.sorted_roi_by_pos_xpath.get(chunk_idx)
if not roi:
raise IndexError(f"Chunk {chunk_idx} does not exist")
return roi.pos_xpath_list
# Usage
paths = get_chunk_source_xpaths(dom, 2)
print("\n".join(paths))
```
### Mapping Chunks to Original HTML Snippets
```python
def chunk_to_html(dom, chunk_idx: int) -> str:
"""Concatenate the HTML of all nodes that belong to the chunk."""
xpaths = get_chunk_source_xpaths(dom, chunk_idx)
snippets = [
dom.tree_representation.xpaths_metadata[x].bs4_elem.prettify()
for x in xpaths
]
return "\n".join(snippets)
print(chunk_to_html(dom, 0))
```
### CLI Verbose Output for Debugging
Use the command-line interface with verbose flags to inspect positional XPaths without writing code:
```bash
# Run the CLI with maximal verbose to see the internal XPaths
cat page.html | betterhtmlchunking --max-length 1500 --maximal-verbose
```
The verbose log prints lines like:
```
ROI 0 → pos_xpath_list=['/html/body/div[1]/h1']
ROI 1 → pos_xpath_list=['/html/body/div[1]/p[1]', '/html/body/div[1]/p[2]']
```
## Key Implementation Files
| File | Role | Link |
|------|------|------|
| [`betterhtmlchunking/tree_representation.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_representation.py) | Builds the DOM tree, computes and stores positional XPaths (`pos_xpaths_list`, `xpaths_metadata`). | [tree_representation.py](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_representation.py) |
| [`betterhtmlchunking/tree_regions_system.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_regions_system.py) | Detects Regions-of-Interest (ROIs) and groups the XPaths that belong to each chunk (`RegionOfInterest.pos_xpath_list`, `sorted_roi_by_pos_xpath`). | [tree_regions_system.py](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_regions_system.py) |
| [`betterhtmlchunking/utils.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/utils.py) | Helper for filtering XPaths (e.g., removing unwanted tags). | [utils.py](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/utils.py) |
| [`README.md`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/README.md) | High-level usage examples and explanation of chunking workflow. | [README.md](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/README.md) |
## Summary
- **Positional XPaths** in `betterhtmlchunking` use indexed notation (e.g., `/html/body/div[2]/p[1]`) to pinpoint exact element locations in the source HTML.
- Access the complete list of document XPaths via `DOMTreeRepresentation.pos_xpaths_list` or map individual XPaths to BeautifulSoup elements using `xpaths_metadata`.
- Trace specific chunks through `TreeRegionsSystem.sorted_roi_by_pos_xpath`, where each `RegionOfInterest` stores its source XPaths in `pos_xpath_list`.
- Retrieve original HTML snippets by combining `pos_xpath_list` with `xpaths_metadata` to access the underlying `bs4_elem` objects.
## Frequently Asked Questions
### What is a positional XPath in betterhtmlchunking?
A positional XPath is a string representation of an element's location in the HTML DOM that includes numeric indices for sibling elements (e.g., `/html/body/div[2]/p[1]`). According to the `betterhtmlchunking` source code in [`tree_representation.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/tree_representation.py), these paths are generated by the `get_pos_xpath_from_bs4_elem()` function which walks the BeautifulSoup parent chain and counts siblings to create deterministic, index-based paths.
### How do I get the original HTML element from a chunk's XPath?
You can retrieve the original BeautifulSoup Tag object by accessing the `xpaths_metadata` dictionary stored in `DOMTreeRepresentation`. Each XPath maps to a `NodeMetadata` object containing the `bs4_elem` attribute. For example: `dom.tree_representation.xpaths_metadata["/html/body/div[1]/p[2]"].bs4_elem` returns the original HTML element, which you can then render using `.prettify()` or process further.
### Can I access positional XPaths without using the CLI?
Yes, positional XPaths are fully accessible through the Python API without invoking the command-line interface. After instantiating `DomRepresentation` and calling `.start()`, you can access `dom.tree_representation.pos_xpaths_list` for all document XPaths or `dom.tree_regions_system.sorted_roi_by_pos_xpath` for chunk-specific XPaths. The CLI's `--maximal-verbose` flag merely provides a convenient way to view these same values in terminal output.
### Where are positional XPaths stored in the source code?
Positional XPaths are primarily stored in two locations within the `betterhtmlchunking` codebase. In [`betterhtmlchunking/tree_representation.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_representation.py), the `DOMTreeRepresentation` class maintains `pos_xpaths_list` (a flat list of all paths) and `xpaths_metadata` (a dictionary mapping paths to element metadata). In [`betterhtmlchunking/tree_regions_system.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_regions_system.py), the `RegionOfInterest` class stores `pos_xpath_list` containing only the XPaths belonging to that specific chunk, while `TreeRegionsSystem` organizes these into `sorted_roi_by_pos_xpath`.