How book-to-skill Detects and Handles Scanned PDFs: A Technical Deep Dive

The book-to-skill library detects scanned PDFs by running a fast pre-flight check using pdftotext on the first five pages; if no text is extracted, the parser returns None to skip expensive processing and avoid pipeline failures.

The virgiliojr94/book-to-skill project provides intelligent document parsing for skill extraction, but scanned image-only PDFs present a unique challenge since they contain no selectable text. Understanding how the system detects and handles scanned PDFs is critical for developers building robust extraction pipelines. The implementation relies on lightweight external utilities and defensive programming to maintain performance when encountering rasterized documents.

Fast Pre-Flight Detection with looks_image_only()

The detection mechanism centers on the looks_image_only() function defined in book_to_skill/parsers/pdf.py at lines 93-100. This utility performs a lightweight smoke test before committing to full text extraction.

The function invokes the external tool pdftotext on the first N pages of the document, defaulting to five pages. It executes the subprocess and evaluates result.stdout.strip(). If the output is empty, the PDF is immediately flagged as image-only—the classic signature of a scanned document containing only raster images without embedded text layers.

This approach leverages a well-tested external utility rather than loading heavy Python libraries, keeping the detection overhead minimal even on large documents.

Graceful Fallback Handling

When looks_image_only() identifies a scanned PDF, the system triggers a graceful fallback that prevents wasted computation. Instead of proceeding through the normal text-extraction chain—which includes extract_with_pdftotext(), extract_with_pypdf(), and extract_with_pdfminer()—the parser simply returns None.

This early return allows the higher-level extraction logic in scripts/extract.py to skip the file entirely and proceed to the next source. The design intentionally avoids raising exceptions, ensuring that encountering a scanned document does not halt batch processing or trigger error cascades in the pipeline.

Final Empty-Text Guard

Even if the pre-flight detection is skipped—for example, when pdftotext is not installed on the host system—the pipeline implements a final empty-text guard as a safety net. Located in scripts/extract.py, this validation step runs after the generic extraction attempt completes.

If the resulting text string is empty, the system treats the PDF as a failed extraction and omits it from the generated skill output. This redundant check ensures that scanned PDFs cannot pollute the downstream processing steps even when the primary detection mechanism is unavailable.

Implementation Example

Developers can leverage the detection logic directly to implement custom handling for scanned documents:

from book_to_skill.parsers.pdf import looks_image_only

pdf_path = "docs/scanned_book.pdf"

if looks_image_only(pdf_path):
    print("🔎  Image-only PDF detected – skipping text extraction.")
else:
    # Proceed with normal extraction (pdftotext, pypdf, etc.)

    from book_to_skill.parsers.pdf import extract_with_pdftotext
    text = extract_with_pdftotext(pdf_path)
    if text:
        print("✅  Extraction succeeded.")

This pattern mirrors the repository's internal workflow: a quick looks_image_only call guards the heavier extraction functions, preventing costly operations on documents that cannot yield text.

Key Files in the Detection Pipeline

Several files work together to implement the scanned PDF detection and handling strategy:

• book_to_skill/parsers/pdf.py – Contains the core looks_image_only() implementation at lines 93-100, along with extraction backends and page-count helpers.
• scripts/extract.py – Orchestrates the high-level extraction driver and implements the empty-text guard for documents that bypass pre-flight checks.
• book_to_skill/utils.py – Provides utility helpers used by parsers for sanitizing extracted text.
• tests/test_pdf_page_number_detection.py – Contains tests for PDF page-number detection logic relevant to boilerplate removal post-extraction.

Summary

• Pre-flight scanning via looks_image_only() in book_to_skill/parsers/pdf.py uses pdftotext on the first five pages to identify image-only documents.
• Early termination returns None immediately when scanned PDFs are detected, bypassing expensive extraction functions like extract_with_pdfminer().
• Defensive guards in scripts/extract.py catch edge cases where detection is skipped, ensuring empty results never propagate to skill generation.
• External tool dependency on pdftotext provides lightweight, reliable detection without loading heavy Python PDF libraries.

Frequently Asked Questions

What external tool does book-to-skill use to detect scanned PDFs?

The library relies on pdftotext, a command-line utility from the Poppler suite. The looks_image_only() function spawns a subprocess running pdftotext on the first five pages of the target PDF. If the command returns no stdout content after stripping whitespace, the document is classified as image-only.

What happens when book-to-skill encounters a scanned PDF during batch processing?

When a scanned PDF is detected, the parser returns None rather than attempting full text extraction. This allows the orchestration layer to skip the file silently and continue processing remaining documents. No exceptions are raised, ensuring batch jobs complete successfully even when encountering rasterized content.

Can the scanned PDF detection be bypassed if pdftotext is unavailable?

Yes. If pdftotext is not installed, the pre-flight check is skipped and the system proceeds to standard extraction methods. However, the empty-text guard in scripts/extract.py serves as a backup, catching the resulting empty string and marking the extraction as failed before the document reaches the skill generation phase.

Where is the primary logic for detecting image-only PDFs implemented?

The core detection logic resides in book_to_skill/parsers/pdf.py at lines 93-100. This file contains the looks_image_only() function that implements the pdftotext-based check, alongside various text extraction backends that are conditionally invoked based on the detection results.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}