PDF Extraction Modes in book-to-skill: 4 Methods Explained

The book-to-skill repository implements four distinct PDF extraction modes—layout-aware pdftotext, pure-Python pypdf, pdfminer.six, and Docling—that range from fast binary extraction to layout-preserving document conversion, all unified by a shared clean_pdftotext post-processing pipeline.

The virgiliojr94/book-to-skill project provides a flexible PDF parsing architecture in book_to_skill/parsers/pdf.py designed to handle everything from simple text-based documents to complex technical books with tables and code blocks. Understanding the available PDF extraction modes helps you select the optimal strategy based on your environment constraints, available system dependencies, and document complexity.

The Four PDF Extraction Modes

The extraction logic in book_to_skill/parsers/pdf.py implements a cascading fallback system that attempts tools in order of efficiency, while also allowing explicit selection for specialized use cases.

Layout-Aware pdftotext (Poppler)

extract_with_pdftotext is the preferred method when the Poppler pdftotext binary is available in the system PATH (detected via shutil.which("pdftotext")).

This mode executes the external binary with the -layout flag to preserve page-wise formatting, then pipes the output through clean_pdftotext (lines 30-73) to strip repeated headers, footers, page numbers, and hyphen-wrapped words. It offers the best balance of speed and layout fidelity for well-structured, text-based PDFs.

Pure-Python pypdf

extract_with_pypdf serves as the first fallback when the Poppler binary is absent. This mode uses the pypdf.PdfReader class to iterate over pages and extracts text via page.extract_text().

While slower than the binary approach for large documents, this method requires no external system dependencies. Like the pdftotext mode, it applies the same clean_pdftotext post-processing to ensure consistent de-hyphenation and boilerplate removal across all extraction strategies.

pdfminer.six

extract_with_pdfminer activates when both pdftotext and pypdf are unavailable or fail. It calls pdfminer.high_level.extract_text, which returns form-feed delimited pages and handles complex PDF encodings that pypdf might miss.

This mode is particularly useful for PDFs with non-standard text encodings or embedded fonts where other extractors return garbled output. It also feeds results through the shared clean_pdftotext pipeline for consistency.

Docling (Layout-Preserving)

extract_with_docling is the most sophisticated mode, designed explicitly for technical books with complex layouts, tables, and code blocks. Unlike the text-only extractors, Docling instantiates a DocumentConverter with PDF-specific options (do_ocr=False, do_table_structure=True).

This mode generates Markdown output that preserves structural elements like tables and code blocks through docling's document model. However, it requires the heavier docling dependency and is typically invoked explicitly via CLI flags rather than selected automatically by the fallback chain.

Shared Post-Processing Pipeline

Regardless of which PDF extraction mode produces the raw text, all four methods route output through clean_pdftotext in book_to_skill/parsers/pdf.py (lines 30-73). This utility performs two critical normalization steps:

• Header/Footer Removal: Detects and strips repeated page-edge content using _PDF_PAGE_NUM patterns and boiler-plate detection heuristics.
• Hyphenation Repair: Joins words broken by hyphenation across line breaks using the _PDF_HYPHEN_WRAP pattern.

If none of the extraction tools are present in the environment, the pipeline returns None, triggering the higher-level code to fall back to an empty-text guard.

How to Use Each Extraction Mode

Automatic Selection via CLI

The command-line interface in book_to_skill/cli.py automatically selects the best available extractor:

python -m book_to_skill extract my_book.pdf > output.md

The CLI checks for pdftotext availability first, then falls back through pypdf and pdfminer.six automatically.

Direct Function Calls

For precise control over the PDF extraction mode, import specific extractors from book_to_skill/parsers/pdf.py:

from book_to_skill.parsers.pdf import (
    extract_with_pdftotext,
    extract_with_pypdf,
    extract_with_pdfminer,
    extract_with_docling,
)

pdf_path = "technical_manual.pdf"

# Prefer system binary for speed

text = extract_with_pdftotext(pdf_path)

# Cascade through fallbacks

if text is None:
    text = extract_with_pypdf(pdf_path) or \
           extract_with_pdfminer(pdf_path)

Explicit Docling for Technical Documents

When processing technical books where table layout fidelity is crucial, bypass the automatic chain and invoke Docling directly:

from book_to_skill.parsers.pdf import extract_with_docling

markdown_output = extract_with_docling("complex_book.pdf")
if markdown_output:
    print(markdown_output[:500])

Summary

• Layout-aware pdftotext provides the fastest extraction with good layout preservation, requiring the Poppler system binary.
• pypdf offers a pure-Python fallback with zero system dependencies, suitable for simple text-based PDFs.
• pdfminer.six handles complex encodings and non-standard PDFs that other extractors might fail to parse.
• Docling preserves structural elements like tables and code blocks through Markdown conversion, ideal for technical documentation.
• All modes share the clean_pdftotext post-processor (lines 30-73 of book_to_skill/parsers/pdf.py) to normalize headers, footers, and hyphenation.

Frequently Asked Questions

Which PDF extraction mode is fastest for simple documents?

Layout-aware pdftotext is the fastest option for simple, text-based PDFs because it executes an optimized C binary rather than parsing in Python. According to the source code in book_to_skill/parsers/pdf.py, this mode is checked first via shutil.which("pdftotext") and should be preferred when available.

When should I use Docling instead of the default extractors?

Use extract_with_docling when your PDF contains complex layouts, tables, code blocks, or mixed content that requires structural preservation. While the default extractors return plain text, Docling's DocumentConverter with do_table_structure=True generates Markdown that maintains the document's visual hierarchy, making it essential for technical books and research papers.

What happens if no PDF extraction tools are installed?

If neither pdftotext, pypdf, pdfminer.six, nor docling are available, the extraction functions in book_to_skill/parsers/pdf.py return None. The higher-level code in book_to_skill/cli.py then triggers an empty-text guard, preventing downstream processing errors while alerting you to the missing dependencies.

How does book-to-skill handle hyphenated words across line breaks?

All PDF extraction modes feed raw text through the clean_pdftotext function (lines 30-73), which uses the _PDF_HYPHEN_WRAP regex pattern to detect and join hyphenated words that span line endings. This ensures that words split by PDF layout engines are reconstructed into complete tokens before further processing.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}