betterhtmlchunking
BetterHTMLChunking is a Python library for intelligent HTML segmentation. It builds a DOM tree from raw HTML and extracts content-rich regions of interest, making content analysis effortless. Great for LLM based processing.
Learn how order_regions_of_interest_by_pos_xpath sorts HTML chunks by document position using XPath document indexes. Ensure your chunks maintain original HTML visual flow.
How to Use the --text-only Flag in betterhtmlchunking for Plain Text OutputLearn to use the --text-only flag in betterhtmlchunking to extract clean plain text output without HTML markup. Simplify your text processing effortlessly.
How MAX_NODE_REPR_LENGTH Controls Chunk Count and Semantic Coherence in BetterHTMLChunkingDiscover how MAX_NODE_REPR_LENGTH impacts chunk count and semantic coherence in BetterHTMLChunking. Optimize your DOM parsing for better results.
How to Process HTML from Stdin Using the BetterHTMLChunking CLI and Pipe Output to Other ToolsProcess HTML from stdin using the BetterHTMLChunking CLI. Pipe output to tools like jq and grep for powerful text manipulation. Integrate seamlessly with Unix pipes.
How BetterHTMLChunking Preserves Parent-Child Relationships and Hierarchical Context in HTML ChunksDiscover how BetterHTMLChunking maintains hierarchical context in HTML chunks. Learn how it uses XPath and BeautifulSoup to preserve parent-child relationships for better data integrity.
betterhtmlchunking CLI Logging Modes: Default vs Verbose vs Maximal-VerboseUnderstand betterhtmlchunking CLI logging modes. Learn the differences between default, verbose, and maximal-verbose to control your HTML chunking diagnostics and troubleshoot effectively.
How to Filter Chunks by Size Using the JSON Output and jq in betterhtmlchunkingFilter betterhtmlchunking JSON output by size using jq. Learn to select chunks based on html_length or text_length for efficient data processing.
How Root XPath Detection Works in betterhtmlchunking When the /html Element Is MissingDiscover how betterhtmlchunking performs root XPath detection even without a /html element. Learn about its fallback mechanism for robust HTML fragment processing.
How to Use the betterhtmlchunking tree_representation API to Retrieve Node Metadata (text_length and html_length)Learn to use the betterhtmlchunking tree_representation API to access node metadata like text_length and html_length via the xpaths_metadata dictionary. Unlock valuable DOM information.
Internal Architecture of tree_regions_system: Queue-Based Region Detection in BetterHTMLChunkingDiscover the internal architecture of the tree_regions_system in betterhtmlchunking. Learn how queue-based processing partitions HTML documents efficiently while preserving order.
How to Combine betterhtmlchunking with LangChain for Hybrid RAG PipelinesIntegrate betterhtmlchunking and LangChain for hybrid RAG pipelines. Learn to create structure-aware chunks from HTML ARoIs for advanced retrieval-augmented generation workflows.
How the html_unescape Parameter Handles HTML Entities in BetterHTMLChunkingUnderstand how the html_unescape parameter in BetterHTMLChunking decodes HTML entities before DOM parsing. Learn to control HTML entity handling for cleaner content processing.
Have a question about this repo?
These articles cover the highlights, but your codebase questions are specific. Give your agent direct access to the source. Share this with your agent to get started:
curl -s "https://instagit.com/install.md" 
Maintain an open-source project? Get it listed too →