How to Use the --text-only Flag in betterhtmlchunking for Plain Text Output

Pass the --text-only flag to the betterhtmlchunking chunk command to emit clean, markup-free text instead of HTML by selecting pre-computed plain-text chunks from RenderSystem.text_render_roi.

The betterhtmlchunking library intelligently splits HTML documents into manageable chunks while preserving semantic structure. When you need to extract plain text output without HTML markup, the --text-only flag provides a direct path to clean content by leveraging the library's dual rendering system implemented in betterhtmlchunking/cli.py.

CLI Parsing and Flag Definition

The --text-only option is defined in betterhtmlchunking/cli.py using the Typer framework. It is declared as a boolean option defaulting to False at lines 61-65:

text_only: bool = typer.Option(
    False,
    "--text-only",
    help="Output text content only (no HTML markup)",
)

When the user activates this flag, Typer stores the Boolean in the text_only variable. The CLI later consults this variable to determine whether to return content from RenderSystem.text_render_roi (plain text) or RenderSystem.html_render_roi (HTML).

Three Output Modes Affected by --text-only

The flag influences behavior across three distinct output paths in betterhtmlchunking/cli.py, though it is ignored when exporting to JSON.

Single Chunk Output (Default)

When processing a single chunk without the --all-chunks flag, setting --text-only causes the CLI to print the plain-text version from text_render_roi directly to stdout. This logic is implemented at lines 167-171 in cli.py.

Batch Export with --all-chunks

When using the --all-chunks flag combined with --text-only, the tool writes each chunk to disk with a .txt extension instead of .html. This behavior occurs at lines 144-150 in cli.py, where the output path construction checks the text_only variable to determine the appropriate file suffix.

JSON Format Exception

The --text-only flag is ignored when using --format json. According to the source code at lines 98-115 in cli.py, JSON output always includes both html and text fields regardless of the flag setting, providing complete data for applications that require both representations.

The Rendering Pipeline Behind the Flag

The dual-output capability originates in betterhtmlchunking/render_system.py. When DomRepresentation.start() (in main.py) initializes the pipeline, it creates a RenderSystem instance that pre-computes both representations for every region of interest (ROI) during the initial parsing phase.

The rendering process works as follows:

1. HTML rendering: Uses BeautifulSoup's prettify() method with formatter="minimal" to generate clean HTML.
2. Text rendering: Calls parsel_text.get_bs4_soup_text() to strip all markup and extract readable text content.

These results are stored in parallel dictionaries at lines 73-80 and 84-90 in render_system.py:

• html_render_roi – full HTML of each chunk
• text_render_roi – plain-text counterpart

Because both versions are generated during the initial parsing phase, the --text-only flag incurs zero additional overhead—it simply selects the pre-computed text dictionary at output time.

Practical Code Examples

Replace betterhtmlchunking with your actual entry-point name if you installed the package under a different command.

Print the first chunk as plain text:

cat page.html | betterhtmlchunking chunk --text-only

Print a specific chunk (e.g., chunk 3) as plain text:

cat page.html | betterhtmlchunking chunk --chunk-index 3 --text-only

Export all chunks as .txt files:

mkdir chunks
cat page.html | betterhtmlchunking chunk \
    --all-chunks --output-dir chunks --text-only

Combine with other options:

cat page.html | betterhtmlchunking chunk \
    --max-length 5000 --text-only --verbose

Summary

• The --text-only flag in betterhtmlchunking outputs clean, markup-free text by selecting from pre-computed text_render_roi data stored during the initial parsing phase.
• Defined in betterhtmlchunking/cli.py as a Typer boolean option, it affects single-chunk stdout output and batch .txt file generation when using --all-chunks.
• The flag is ignored for JSON output, which always contains both html and text fields to provide complete data representations.
• Text extraction occurs in betterhtmlchunking/render_system.py using parsel_text.get_bs4_soup_text(), with both HTML and text versions cached in parallel dictionaries during document processing.

Frequently Asked Questions

Does the --text-only flag remove all HTML entities and tags?

Yes. When --text-only is active, the tool returns content from RenderSystem.text_render_roi, which is generated by parsel_text.get_bs4_soup_text() in betterhtmlchunking/render_system.py. This function strips all HTML tags and entities, returning only the human-readable text content of each chunk.

Can I use --text-only with JSON output?

No. According to the source code in betterhtmlchunking/cli.py (lines 98-115), the --text-only flag is explicitly ignored when using --format json. JSON output always includes both html and text fields to provide complete data for downstream applications that may require both representations.

What file extension does --text-only use when exporting chunks?

When combined with the --all-chunks flag, --text-only causes the tool to write files with the .txt extension instead of .html. This behavior is implemented in betterhtmlchunking/cli.py (lines 144-150), where the output path construction checks the text_only variable to determine the appropriate suffix.

Does using --text-only slow down processing?

No. The --text-only flag incurs zero additional overhead because both HTML and plain-text versions of every chunk are pre-computed during the initial parsing phase in RenderSystem (betterhtmlchunking/render_system.py). The flag simply selects which pre-computed dictionary (text_render_roi versus html_render_roi) to use at output time.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}