How book-to-skill Handles Different PDF Types: Technical vs Text-Heavy Extraction Explained

Book-to-skill uses a layered extraction pipeline that adapts to document structure through two distinct modes—technical for layout-rich documents and text for simple prose—automatically falling back to robust alternatives when primary methods fail.

The book-to-skill repository by virgiliojr94 implements an intelligent PDF processing system that recognizes not all PDFs are created equal. Technical books with code blocks and tables require fundamentally different handling than text-heavy novels. The project solves this through mode-based extraction with graceful degradation.

---

Detecting Document Type Before Extraction

Before committing resources to full parsing, book-to-skill performs a lightweight sanity check. The looks_image_only function in parsers/pdf.py (lines 94-107) samples the first N pages using pdftotext to detect scanned PDFs.

# From book_to_skill/parsers/pdf.py

def looks_image_only(file_path: Path, pages_to_check: int = 5) -> bool:
    """
    Quick check: if pdftotext returns nothing on first N pages,
    assume this is a scanned/image-based PDF requiring OCR.
    """

If no text is detected, the pipeline raises an ExtractionError prompting the user to OCR the document first. This prevents wasted computation on image-only PDFs.

---

The Two Extraction Modes: Technical vs Text

The CLI exposes this functionality through the --mode flag, parsed in utils.py (lines 46-52):

# Force technical mode for programming books, research papers

book-to-skill programming_book.pdf --mode technical

# Use text mode for novels, essays (default behavior)

book-to-skill novel.pdf --mode text

Technical Mode: Layout-Aware Extraction with Docling

When extraction_mode == "technical", the pipeline prioritizes extract_with_docling in parsers/pdf.py (lines 145-168). This leverages the Docling library to preserve:

• Multi-column layouts
• Tables and structured data
• Code blocks with indentation
• Figure captions and complex formatting

# Simplified from book_to_skill/parsers/pdf.py

def extract_with_docling(input_path: Path) -> str:
    try:
        from docling.document_converter import DocumentConverter
        converter = DocumentConverter()
        result = converter.convert(input_path)
        return result.document.export_to_markdown()
    except Exception:
        return ""  # Signals fallback to text mode

If Docling isn't installed or returns empty text, the pipeline automatically degrades to text mode. This fail-forward design ensures extraction always succeeds when possible.

---

Text Mode: The Three-Tool Fallback Chain

For text-heavy PDFs or when technical mode fails, extract_single_file in utils.py (lines 70-88) orchestrates a cascading fallback:

Priority	Tool	Purpose
1	pdftotext (-layout flag)	Fast C++ binary, handles most standard PDFs
2	pypdf	Pure-Python fallback when binaries unavailable
3	pdfminer.six	Lowest-level parser for malformed PDFs

# From book_to_skill/utils.py - simplified extraction logic

def extract_single_file(input_path, extraction_mode, install_mode):
    if looks_image_only(input_path):
        raise ExtractionError("Scanned PDF – OCR first")
    
    if extraction_mode == "technical":
        text = extract_with_docling(input_path)
        if not text:
            extraction_mode = "text"  # Degrade gracefully

    
    if extraction_mode == "text":
        text = (
            extract_with_pdftotext(input_path) or
            extract_with_pypdf(input_path) or
            extract_with_pdfminer(input_path)
        )
    return text

The or-chain stops at the first successful extraction, optimizing for speed while maintaining reliability.

---

Post-Processing: Cleaning Extracted Text

Regardless of extraction path, raw text passes through clean_pdftotext in parsers/pdf.py (lines 30-73). This function:

• Strips repeated headers/footers running across pages
• Removes isolated page numbers
• Merges words split by hyphenation across line breaks
• Normalizes whitespace

Additional sanitization in sanitize.py handles invisible Unicode characters that could corrupt downstream processing.

---

Metadata Extraction: Page Counting

Independent of text extraction, count_pages (lines 71-90 in parsers/pdf.py) reports total page count using pdfinfo or pypdf. This provides:

• Progress indicators during processing
• Metadata for output files
• User-facing statistics about document size

---

Summary

• book-to-skill handles different PDF types through explicit mode selection (--mode technical|text) with automatic fallback
• Technical mode uses Docling for layout-aware extraction of complex documents
• Text mode cascades through three tools (pdftotext → pypdf → pdfminer) for speed and reliability
• Image-only PDFs are caught early via looks_image_only to avoid wasted processing
• All extractions are cleaned via clean_pdftotext to normalize formatting artifacts

---

Frequently Asked Questions

How does book-to-skill detect if a PDF needs OCR?

The looks_image_only function samples the first 5 pages with pdftotext. If zero text is extracted, it classifies the PDF as image-only and raises an error directing users to OCR first. This check runs in parsers/pdf.py lines 94-107.

What happens if Docling isn't installed in technical mode?

The extract_with_docling function catches import or execution errors and returns an empty string. The caller in extract_single_file detects this empty result and automatically switches to text mode, falling back to the pdftotext chain.

Why not always use Docling for every PDF?

Docling is slower and overkill for simple text documents. Novels and essays extract faster with pdftotext while producing cleaner output. The mode system lets users optimize for their specific document type.

Can the extraction tools be used programmatically?

Yes. All extraction functions in parsers/pdf.py accept Path objects and return strings. Import directly: from book_to_skill.parsers.pdf import extract_with_docling, extract_with_pdftotext.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}