How Book-to-Skill Extracts PDF Files: A Complete Guide to the Multi-Backend Pipeline

Book-to-Skill extracts PDF files through a layered pipeline that detects image-only documents, selects the optimal text-extraction backend based on user mode, and falls back through three successive tools when needed.

Book-to-Skill is an open-source tool that converts technical books into structured skill data. Understanding how it extracts PDF files reveals a robust engineering approach designed to handle diverse document formats—from clean digital PDFs to scanned images. The extraction logic resides primarily in book_to_skill/parsers/pdf.py and book_to_skill/utils.py, with extract_single_file() serving as the central orchestrator.

Image-Only Detection Before Extraction

The pipeline begins with defensive validation. The looks_image_only() function in book_to_skill/parsers/pdf.py#L93-L101 runs a quick pdftotext probe on the first few pages to detect scanned-image PDFs that contain no extractable text.

If no text is found, extraction halts immediately and the user receives a clear prompt to run OCR first. This prevents wasted processing time on unextractable documents and ensures users understand why a PDF cannot be parsed.

# Example: Detect whether a PDF is image-only before extraction

from book_to_skill.parsers.pdf import looks_image_only

if looks_image_only("scanned.pdf"):
    print("Run OCR first – the PDF contains no extractable text.")
else:
    print("Proceed with text extraction.")

Technical Mode: Layout-Aware Extraction with Docling

When the user specifies --mode technical, Book-to-Skill prioritizes Docling for PDF extraction. Docling preserves structural elements like tables and code blocks that plain-text extraction would destroy.

The fallback logic in book_to_skill/utils.py#L58-L66 attempts Docling first. If Docling is unavailable or returns empty output, the pipeline automatically degrades to plain-text mode without user intervention.

# Example: Extract a PDF in technical mode (layout-aware)

from book_to_skill.utils import extract_single_file
from pathlib import Path

pdf_path = Path("example.pdf")
result = extract_single_file(pdf_path, extraction_mode="technical", install_mode="ask")
print(result["extraction_method"])   # → "docling" (or fallback to "pdftotext")

print(result["text"][:200])          # first 200 characters of extracted text

Plain-Text Mode: The Three-Backend Fallback Chain

When technical mode is inactive or Docling fails, Book-to-Skill executes a three-stage fallback chain in book_to_skill/parsers/pdf.py. Each backend is attempted only if the previous one produces no text:

1. pdftotext (Preferred)

The extract_with_pdftotext() function at book_to_skill/parsers/pdf.py#L76-L88 calls the external pdftotext -layout -enc UTF-8 utility. This preserves basic layout and handles most well-formed PDFs efficiently.

Output passes through clean_pdftotext() at book_to_skill/parsers/pdf.py#L30-L73, which strips:

• Repeated headers and footers
• Page numbers
• Hyphenation artifacts at line breaks

2. pypdf (First Fallback)

If pdftotext fails or returns empty, extract_with_pypdf() at book_to_skill/parsers/pdf.py#L12-L25 uses the pure-Python pypdf library. This eliminates external dependencies but may handle complex layouts less gracefully.

3. pdfminer.six (Last Resort)

The final fallback, extract_with_pdfminer() at book_to_skill/parsers/pdf.py#L33-L42, calls pdfminer.high_level.extract_text(). This handles malformed PDFs that break other parsers, though it is slower.

# Example: Force plain-text extraction, manually invoking the fallback chain

from book_to_skill.parsers.pdf import extract_with_pdftotext, extract_with_pypdf, extract_with_pdfminer

text = extract_with_pdftotext("example.pdf")
if not text:
    text = extract_with_pypdf("example.pdf")
if not text:
    text = extract_with_pdfminer("example.pdf")
print(text[:300])  # preview

Exhaustion of all three backends triggers an ExtractionError at book_to_skill/utils.py#L92-L99 with actionable guidance on installing required dependencies.

Page Counting with Progressive Fallbacks

After successful text extraction, count_pages() in book_to_skill/parsers/pdf.py#L71-L111 determines document length using progressively safer methods:

• Primary: pdfinfo CLI for fast, accurate metadata
• Secondary: pypdf page enumeration
• Tertiary: Counting form-feed characters left by pdfminer

This ensures reliable page counts even when external tools are missing.

Post-Extraction Processing

The extracted text undergoes additional refinement before final output:

• Sanitization: sanitize_extracted_text() at book_to_skill/utils.py#L41-L53 strips invisible Unicode control characters that could corrupt downstream processing
• Structure detection: detect_structure() at book_to_skill/utils.py#L54-L60 identifies chapters and table-of-contents patterns
• Metadata assembly: Token estimation and extraction method tracking complete the JSON result structure

Key Implementation Files

File	Purpose
book_to_skill/parsers/pdf.py	All PDF extraction backends, image detection, page counting
book_to_skill/utils.py	Orchestration logic, mode selection, sanitization, metadata
scripts/extract.py	CLI entry point

Summary

• Book-to-Skill extracts PDF files through image detection → mode selection → backend fallback → sanitization
• Technical mode prefers Docling for layout preservation; plain-text mode chains pdftotext → pypdf → pdfminer.six
• The looks_image_only() guard prevents wasted processing on scanned-image PDFs
• All extraction outputs pass through clean_pdftotext() and sanitize_extracted_text() for consistent quality
• extract_single_file() in book_to_skill/utils.py#L50-L84 coordinates the entire pipeline

Frequently Asked Questions

What happens if no PDF extraction backend is installed?

Book-to-Skill raises an ExtractionError with specific installation instructions for pdftotext, pypdf, or pdfminer.six. The error message guides users to resolve the dependency gap rather than failing silently.

Why does technical mode fall back to plain-text extraction?

Docling requires additional dependencies and may fail on certain PDF structures. The fallback in book_to_skill/utils.py#L58-L66 ensures users always receive extracted text rather than a hard failure, prioritizing content retrieval over formatting fidelity.

How does Book-to-Skill handle scanned PDFs without OCR text?

The looks_image_only() function detects these during preflight checks and stops processing with a clear user message. The tool does not perform OCR itself—users must preprocess scanned documents with external OCR tools before ingestion.

Can I force a specific extraction backend?

Yes. While extract_single_file() automates selection, you can import individual functions from book_to_skill.parsers.pdf to call extract_with_pdftotext(), extract_with_pypdf(), or extract_with_pdfminer() directly, as shown in the code examples above.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}