# How the html_unescape Parameter Handles HTML Entities in BetterHTMLChunking > Understand how the html_unescape parameter in BetterHTMLChunking decodes HTML entities before DOM parsing. Learn to control HTML entity handling for cleaner content processing. - Repository: [Carlos A. Planchón/betterhtmlchunking](https://github.com/carlosplanchon/betterhtmlchunking) - Tags: how-to-guide - Published: 2026-02-26 --- **The `html_unescape` parameter is a boolean flag that controls whether HTML entities like `<` and `&` are decoded into literal characters (`<`, `&`) before the DOM parsing and chunking pipeline processes the input content.** The `html_unescape` parameter is a critical configuration option in the **BetterHTMLChunking** library (carlosplanchon/betterhtmlchunking) that determines how encoded characters are processed during initial content ingestion. When enabled, this parameter ensures that HTML entities are converted to their literal equivalents, allowing downstream chunking algorithms to work with clean markup rather than encoded strings. ## What is the html_unescape Parameter? The `html_unescape` parameter is a boolean attribute of the `DomRepresentation` class that defaults to `True`. It governs whether the input HTML string undergoes entity decoding via Python's standard library `html.unescape()` function before being stored and processed. This setting affects how special characters appear in the final chunked output and influences length calculations throughout the pipeline. ## Source Code Location and Default Configuration In the BetterHTMLChunking source code, the `html_unescape` attribute is defined in [`betterhtmlchunking/main.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/main.py) at lines 54-57 using the `attrs` library: ```python html_unescape: bool = attrs.field( validator=type_validator(), default=True ) ``` This declaration establishes `True` as the default behavior, meaning HTML entities are automatically decoded unless explicitly disabled by the user. ## How the Unescaping Mechanism Works The actual entity decoding occurs during the `__attrs_post_init__` method of the `DomRepresentation` class (lines 71-75 in [`betterhtmlchunking/main.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/main.py)). When `html_unescape` is `True`, the raw HTML content passes through `html.unescape()`: ```python if self.html_unescape: self.website_code = html.unescape(self.website_code) ``` This standard library function converts common HTML entities such as `<` to `<`, `>` to `>`, `&` to `&`, and `"` to `"`. The transformation happens in-place, replacing the encoded string stored in `self.website_code` with its decoded equivalent before any DOM parsing begins. ## Impact on the Chunking Pipeline The state of the `html_unescape` parameter directly influences downstream processing in several critical ways: - **Node Representation Accuracy**: When unescaping is enabled, length calculations performed by `ReprLengthComparisionBy.HTML_LENGTH` count the actual characters rather than the longer entity strings. This produces more accurate size estimates for chunking boundaries. - **Final Output Quality**: Chunks rendered through the `RenderSystem` contain literal characters rather than encoded entities, resulting in cleaner HTML output that is easier to read and process by subsequent systems. - **Preservation of Original Encoding**: When set to `False`, the pipeline operates on the raw entity-encoded strings, preserving the exact input format throughout the chunking process. This is useful when maintaining entity encoding is required for downstream XML processors or specific security policies. ## Practical Configuration Examples ### Enabling Unescaping (Default Behavior) By default, `html_unescape` is `True`, so entities are automatically decoded: ```python from betterhtmlchunking.main import DomRepresentation from betterhtmlchunking.tree_regions_system import ReprLengthComparisionBy raw_html = "

Price: <$100> & tax

" dom = DomRepresentation( MAX_NODE_REPR_LENGTH=200, website_code=raw_html, repr_length_compared_by=ReprLengthComparisionBy.HTML_LENGTH ) # Entities are decoded: "<$100> & tax" assert "<$100>" in dom.website_code assert "<$100>" not in dom.website_code ``` ### Disabling Unescaping To preserve HTML entities in their encoded form, explicitly set the parameter to `False`: ```python dom = DomRepresentation( MAX_NODE_REPR_LENGTH=200, website_code=raw_html, repr_length_compared_by=ReprLengthComparisionBy.HTML_LENGTH, html_unescape=False # turn it off ) # Original encoding is preserved assert "<$100>" in dom.website_code assert "<$100>" not in dom.website_code ``` ### Command-Line Interface Usage The `betterhtmlchunking` CLI exposes this parameter via the `--html-unescape` flag: ```bash # Disable unescaping via CLI betterhtmlchunking --html-unescape false input.html # Enable unescaping (default) betterhtmlchunking input.html ``` Internally, the CLI passes this boolean value directly to the `DomRepresentation` constructor in [`betterhtmlchunking/cli.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/cli.py). ## Summary - The `html_unescape` parameter is a boolean attribute of the `DomRepresentation` class defined in [`betterhtmlchunking/main.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/main.py) that defaults to `True`. - When enabled, it invokes Python's `html.unescape()` during `__attrs_post_init__` to convert HTML entities like `<`, `>`, and `&` into their literal characters. - This affects downstream chunking accuracy by ensuring length calculations and rendered output use actual characters rather than encoded entity strings. - Users can disable the parameter via the constructor or CLI (`--html-unescape false`) to preserve original entity encoding throughout the pipeline. ## Frequently Asked Questions ### What happens if html_unescape is set to False? When `html_unescape` is set to `False`, the input HTML string remains in its original entity-encoded form throughout the entire chunking pipeline. The `DomRepresentation` class skips the `html.unescape()` call in `__attrs_post_init__`, meaning entities like `<` and `&` persist into the DOM parsing and rendering stages. This preserves the exact input format but may cause length calculations to count entity strings rather than the characters they represent. ### Does html_unescape affect the final output of chunked HTML? Yes, the `html_unescape` parameter directly influences the content of rendered HTML chunks. When enabled (the default), entities are decoded before processing, so the `RenderSystem` outputs chunks containing literal characters like `<` and `&`. When disabled, the encoded entities remain in the final output, producing chunks that retain the original `<` and `&` strings. This distinction is critical when downstream systems expect either raw HTML or entity-encoded XML. ### What HTML entities does the html_unescape parameter decode? The `html_unescape` parameter leverages Python's standard library `html.unescape()` function, which decodes all standard HTML5 entities including named entities (e.g., `<`, `>`, `&`, `"`, `'`), decimal numeric entities (e.g., `<`), and hexadecimal numeric entities (e.g., `<`). This comprehensive decoding ensures that any valid HTML entity in the input is converted to its corresponding Unicode character before the DOM representation is constructed. ### Can I change the html_unescape setting when using the command-line interface? Yes, the BetterHTMLChunking CLI exposes the `html_unescape` parameter through the `--html-unescape` flag. By default, the CLI enables unescaping, but you can disable it by passing `--html-unescape false` followed by your input file path. This flag is forwarded directly to the `DomRepresentation` constructor in [`betterhtmlchunking/cli.py`](https://github.com/carlosplanchon/betterhtmlchunking/blob/main/betterhtmlchunking/cli.py), ensuring consistent behavior between programmatic and command-line usage.