What Parsers Are Included in the Book-to-Skill Extractor: A Complete Format Guide

The Book-to-Skill extractor ships with seven deterministic parsers covering plain text, RTF, PDF, HTML, EPUB, DOCX, and Calibre-compatible formats (MOBI/AZW3), each implemented as a dedicated module in the book_to_skill/parsers package.

The virgiliojr94/book-to-skill repository provides a deterministic extraction pipeline orchestrated by scripts/extract.py. At the core of this pipeline sits a modular parser architecture where each supported document type receives its own specialized conversion logic. Understanding what parsers are included in the Book-to-Skill extractor helps developers select the right back-end for their specific document workflows.

Overview of the Parser Architecture

The extractor delegates all format-specific processing to the book_to_skill/parsers directory. Each parser is responsible for converting its assigned format into normalized plain text that downstream components consume. The package initializer at book_to_skill/parsers/__init__.py exposes these functions, while the main extractor script routes files based on their extensions.

Plain-Text Parser (text.py)

The book_to_skill/parsers/text.py module handles raw text files with robust encoding detection.

• Function: read_text_file()
• Capabilities: Detects Byte Order Marks (BOMs), attempts UTF-8 decoding, and falls back to CP1252 or Latin-1 when necessary
• Use case: Processing .txt files with unknown or legacy encodings without data loss

Rich Text Format Parser (rtf.py)

The book_to_skill/parsers/rtf.py module strips formatting from RTF documents.

• Function: extract_rtf()
• Implementation details: Removes RTF control groups, decodes Unicode escape sequences, and provides a regex-based cleanup fallback when the optional striprtf library is unavailable
• Resilience: Operates gracefully whether or not external dependencies are installed

PDF Parser (pdf.py)

The book_to_skill/parsers/pdf.py module offers the most flexible extraction strategy, implementing multiple back-ends to handle diverse PDF structures.

• Primary functions: extract_with_pdftotext(), extract_with_pypdf(), plus internal fallbacks for pdfminer and docling
• Processing pipeline: Normalizes output through de-hyphenation and boiler-plate removal
• Strategy: Attempts pdftotext first (the Poppler utility), falls back to pure-Python pypdf, then pdfminer, and finally docling for complex layouts

HTML Parser (html.py)

The book_to_skill/parsers/html.py module employs a three-tier fallback system to maximize extraction quality across different HTML sources.

• Function: extract_html_file()
• Extraction hierarchy:
  1. trafilatura – Primary choice for article extraction and noise removal
  2. BeautifulSoup (bs4) – Fallback for malformed HTML or when trafilatura misses content
  3. Minimal stdlib HTML parser – Last-resort fallback using only Python standard library
• Output: Returns clean, readable text regardless of HTML quality

EPUB Parser (epub.py)

The book_to_skill/parsers/epub.py module handles the standard ebook archive format.

• Function: extract_epub()
• Methodology: Unpacks the EPUB XHTML content and delegates chapter-wise extraction to the HTML parser
• Architecture: Treats EPUB as a container of HTML documents, ensuring consistent text normalization across both formats

Microsoft Word Parser (docx.py)

The book_to_skill/parsers/docx.py module processes modern Word documents using the python-docx library.

• Function: extract_docx()
• Content handling: Reads both paragraph text and table content, preserving tab characters to maintain table cell separation
• Scope: Specifically targets the .docx format (Office Open XML)

Calibre-Compatible Format Parser (calibre.py)

The book_to_skill/parsers/calibre.py module extends support to proprietary ebook formats through external tooling.

• Function: extract_with_calibre()
• External dependencies: Requires Calibre's ebook-meta and ebook-convert command-line utilities
• Supported formats: MOBI, AZW3, and other formats Calibre can convert to UTF-8 text
• Behavior: Only functions when Calibre is installed on the host system

How to Use the Parsers Programmatically

While scripts/extract.py provides the high-level entry point, you can import individual parsers for custom workflows:

from book_to_skill.parsers.text import read_text_file
from book_to_skill.parsers.rtf import extract_rtf
from book_to_skill.parsers.pdf import extract_with_pdftotext, extract_with_pypdf
from book_to_skill.parsers.html import extract_html_file
from book_to_skill.parsers.epub import extract_epub
from book_to_skill.parsers.docx import extract_docx
from book_to_skill.parsers.calibre import extract_with_calibre

# 1️⃣ Plain-text with automatic encoding detection

plain = read_text_file("chapter.txt")

# 2️⃣ RTF with optional striprtf backend

rtf_text, rtf_backend = extract_rtf("chapter.rtf")

# 3️⃣ PDF (prefers pdftotext binary, cascades to pypdf)

pdf_text = extract_with_pdftotext("chapter.pdf") or extract_with_pypdf("chapter.pdf")

# 4️⃣ HTML via trafilatura → BeautifulSoup → stdlib

html_text = extract_html_file("chapter.html")

# 5️⃣ EPUB (extracts XHTML chapters internally)

epub_text = extract_epub("book.epub")

# 6️⃣ DOCX preserving table structure with tabs

docx_text = extract_docx("chapter.docx")

# 7️⃣ Calibre formats (requires external Calibre installation)

calibre_text = extract_with_calibre("book.mobi")

Each function returns a string or tuple containing the extracted text, ready for downstream skill extraction or analysis pipelines.

Summary

• Seven deterministic parsers cover the complete document ecosystem: text, RTF, PDF, HTML, EPUB, DOCX, and Calibre formats.
• PDF extraction offers four back-ends (pdftotext, pypdf, pdfminer, docling) with automatic cascading fallbacks.
• HTML processing uses a three-tier strategy prioritizing trafilatura, then BeautifulSoup, then the standard library.
• Encoding resilience is built into the text parser via BOM detection and UTF-8/CP1252/Latin-1 fallbacks.
• External integrations include optional striprtf for RTF and Calibre CLI tools for proprietary ebook formats.

Frequently Asked Questions

What file formats does Book-to-Skill support natively?

The extractor supports plain text (TXT), Rich Text Format (RTF), PDF, HTML, EPUB, Microsoft Word (DOCX), and Calibre-compatible formats such as MOBI and AZW3. Each format maps to a specific module in book_to_skill/parsers/.

How does the PDF parser handle different PDF types?

The PDF parser in book_to_skill/parsers/pdf.py implements a cascading strategy. It first attempts extraction using the external pdftotext binary (Poppler), which handles most standard PDFs efficiently. If that fails or is unavailable, it falls back to pypdf, then pdfminer for complex layouts, and finally docling for modern ML-based extraction.

What happens if optional dependencies like trafilatura or Calibre are missing?

The parsers degrade gracefully. The HTML parser will skip trafilatura and use BeautifulSoup, falling back to the standard library if necessary. The RTF parser provides a regex-based cleanup when striprtf is absent. However, the Calibre parser requires the external ebook-convert binary and will not function without it.

Can I use individual parsers without running the full extractor?

Yes. Each parser in book_to_skill/parsers/ exposes standalone functions that you can import directly. For example, from book_to_skill.parsers.epub import extract_epub allows you to process EPUB files independently of the scripts/extract.py orchestration logic.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}