# How to Enable Maximal-Verbose Logging in BetterHtmlChunking to Debug Chunking Issues
> Debug chunking issues with BetterHtmlChunking by enabling maximal-verbose logging. Inspect DOM node details and ROI formation for enhanced troubleshooting. Learn how to activate detailed logging via CLI or programmatically.
- Repository: [Carlos A. Planchón/betterhtmlchunking](https://github.com/carlosplanchon/betterhtmlchunking)
- Tags: how-to-guide
- Published: 2026-02-26
---
**To enable maximal-verbose logging, pass the `--maximal-verbose` flag when using the CLI or call `set_log_level(logging.DEBUG)` programmatically to activate DEBUG-level output that reveals per-node DOM statistics, ROI formation, and chunk boundaries.**
The `betterhtmlchunking` library splits HTML documents into semantically coherent chunks, but debugging why specific nodes group together or split apart requires visibility into the internal decision pipeline. Enabling maximal-verbose mode exposes the DOM tree structure, node representation lengths, and region-of-interest (ROI) calculations that drive the chunking algorithm.
## What Maximal-Verbose Logging Reveals
When you activate maximal-verbose mode, the library emits **DEBUG** level logs to stderr containing four critical diagnostic layers:
- **Per-node DOM statistics**: Every node’s XPath, HTML length, and text length
- **ROI construction**: Each region-of-interest’s total length and constituent node XPaths
- **Final chunk metrics**: Character counts and boundaries for generated chunks
- **Tree state snapshots**: Depth-indexed views of the DOM via `log_tree_node_states`
According to the source code in [`betterhtmlchunking/main.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/main.py) (lines 13–23), the `DomRepresentation.start` method iterates over all nodes after tree construction and logs these details only when maximal-verbose mode is active. Similarly, [`betterhtmlchunking/tree_regions_system.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/tree_regions_system.py) (lines 22–33) exposes the `log_tree_node_states` method that prints a detailed tree view including depths and lengths when invoked.
## Enabling Debug Mode via the CLI
The simplest way to inspect DOM node details is through the command-line interface defined in [`betterhtmlchunking/cli.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/cli.py) (lines 42–44).
```bash
# Standard chunking (INFO level, minimal output)
echo "Title
Paragraph
" | \
betterhtmlchunking --max-length 1000
# Maximal-verbose: full debug dump to stderr
echo "Title
Paragraph
" | \
betterhtmlchunking --max-length 1000 --maximal-verbose
```
When running with `--maximal-verbose`, you’ll see structured debug output like:
```
DEBUG - betterhtmlchunking.main - Total nodes in DOM tree: 27
DEBUG - betterhtmlchunking.main - Node XPath: /html/body/h1, HTML length: 42, Text length: 11
DEBUG - betterhtmlchunking.main - Node XPath: /html/body/p[1], HTML length: 156, Text length: 89
DEBUG - betterhtmlchunking.tree_regions_system - ROI 0: HTML length 384, Nodes XPaths: ['/html/body/h1', '/html/body/p[1]', '/html/body/p[2]']
DEBUG - betterhtmlchunking.main - Generated chunk 0: HTML length 384
```
All diagnostic messages route to **stderr**, ensuring that stdout remains clean for actual chunk output, JSON results, or piped processing.
## Configuring Maximal-Verbose Logging Programmatically
For Python applications integrating the chunker directly, import the logging configuration utilities from [`betterhtmlchunking/logging_config.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/logging_config.py) and set the level to `logging.DEBUG`.
```python
import logging
from betterhtmlchunking.logging_config import setup_root_logger, set_log_level
from betterhtmlchunking.main import DomRepresentation
from betterhtmlchunking.tree_regions_system import ReprLengthComparisionBy
# Initialize logger and set maximal-verbose level
setup_root_logger(level=logging.WARNING) # baseline
set_log_level(logging.DEBUG) # enable maximal-verbose
html_content = "Heading
Content
"
dom = DomRepresentation(
MAX_NODE_REPR_LENGTH=5000,
website_code=html_content,
repr_length_compared_by=ReprLengthComparisionBy.HTML_LENGTH,
)
# Start processing with maximal_verbose flag to trigger internal diagnostics
dom.start(verbose=False, maximal_verbose=True)
```
The `DomRepresentation.start` method accepts a `maximal_verbose` boolean parameter that, when `True`, triggers the detailed per-node logging loop and invokes `log_tree_node_states` from the `TreeRegionsSystem` class.
## Redirecting Debug Output to a File
By default, `setup_root_logger` (defined in [`betterhtmlchunking/logging_config.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/logging_config.py), lines 26–63) attaches a `StreamHandler` to stderr. To capture debug logs for later analysis without cluttering the console, add a `FileHandler` to the root logger:
```python
import logging
from betterhtmlchunking.logging_config import setup_root_logger
# Setup logger with DEBUG level
logger = setup_root_logger(level=logging.DEBUG)
# Create file handler for persistent logs
file_handler = logging.FileHandler("chunking-debug.log")
formatter = logging.Formatter("%(asctime)s %(levelname)s %(name)s %(message)s")
file_handler.setFormatter(formatter)
# Add handler to capture all debug output
logger.addHandler(file_handler)
# Now all maximal-verbose output writes to chunking-debug.log
# while stderr still receives a copy
```
This configuration is particularly useful when batch-processing large HTML corpora where you need to audit chunking decisions post-hoc.
## Interpreting the Debug Output
Understanding the log stream helps you diagnose specific chunking anomalies:
- **Node XPath entries** reveal exactly which DOM elements the algorithm evaluates
- **HTML length vs. Text length** discrepancies indicate whether inline styles, scripts, or attributes are inflating the character count
- **ROI listings** show how the `TreeRegionsSystem` groups adjacent nodes before chunking them
- **Chunk character counts** at the end of the log confirm whether chunks respect your `MAX_NODE_REPR_LENGTH` constraints
If you see nodes with unexpectedly large HTML lengths in the debug output, check for hidden `