# What Scripts Run the EXTRACTOR Phase of Book-to-Skill: Complete Pipeline Guide

> Discover the seven core scripts powering the EXTRACTOR phase of the Book-to-Skill pipeline. Learn how parsers and sanitization scripts process your content for seamless conversion.

- Repository: [Virgilio Junior/book-to-skill](https://github.com/virgiliojr94/book-to-skill)
- Tags: complete-pipeline-guide
- Published: 2026-08-29

---

**The EXTRACTOR phase of Book-to-Skill is driven by seven primary scripts:** an entry-point wrapper ([`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py)), a CLI orchestrator ([`book_to_skill/cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/cli.py)), format-specific parsers in `book_to_skill/parsers/`, and a text-sanitization engine ([`book_to_skill/sanitize.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/sanitize.py)), supported by configuration and utility modules.

The EXTRACTOR phase transforms source documents—PDF, EPUB, DOCX, RTF, HTML, and plain text—into normalized, searchable content for downstream skill generation. In the `virgiliojr94/book-to-skill` repository, this phase follows a deterministic pipeline architecture that separates parsing logic from CLI concerns.

## Entry Point and CLI Orchestration

The extraction workflow begins with two coordination layers that handle I/O setup and argument parsing.

### scripts/extract.py – The UTF-8 Enforcement Wrapper

Located at [`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py), this module serves as the thin CLI entry point. It forces UTF-8 encoding for standard streams and delegates execution to the package’s internal API. Invoke it directly with:

```bash
python -m scripts.extract path/to/book.pdf

```

### book_to_skill/cli.py – Core Driver Logic

The [`book_to_skill/cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/cli.py) file contains the `main()` function that parses command-line arguments, determines the appropriate document parser based on file extension or user hints, and orchestrates the full extraction flow. It reads the input file, invokes the selected parser, passes the raw string to the sanitizer, and writes the final output to disk.

## Document Parsers in book_to_skill/parsers/

Each parser implements a consistent `parse(path: str) -> str` interface. The CLI dynamically selects the correct implementation from the `book_to_skill/parsers/` directory based on the input format.

- **[`book_to_skill/parsers/pdf.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/parsers/pdf.py)** – Extracts text from PDF files using `pdfminer` or `pymupdf`.
- **[`book_to_skill/parsers/epub.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/parsers/epub.py)** – Walks EPUB packages and concatenates XHTML chapter content.
- **[`book_to_skill/parsers/docx.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/parsers/docx.py)** – Reads Microsoft Word documents via `python-docx`.
- **[`book_to_skill/parsers/html.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/parsers/html.py)** – Strips markup and isolates primary content from HTML files.
- **[`book_to_skill/parsers/rtf.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/parsers/rtf.py)** – Handles Rich Text Format decoding.
- **[`book_to_skill/parsers/text.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/parsers/text.py)** – Passes through raw `.txt` files unchanged.
- **[`book_to_skill/parsers/calibre.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/parsers/calibre.py)** – Provides optional Calibre-based conversion for obscure or legacy formats.

## Text Sanitization and Utilities

After raw extraction, the pipeline normalizes content through dedicated processing modules.

### book_to_skill/sanitize.py – Post-Extraction Cleaning

The `sanitize_text()` function in [`book_to_skill/sanitize.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/sanitize.py) normalizes Unicode forms, removes problematic control characters, and strips boilerplate artifacts that commonly appear in extracted text.

### book_to_skill/utils.py – Shared Infrastructure

[`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py) supplies generic helpers for path manipulation and logging used across the CLI and parser modules.

## Configuration and Error Handling

- **[`book_to_skill/config.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/config.py)** – Stores default options including output directory paths and language settings.
- **[`book_to_skill/exceptions.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/exceptions.py)** – Defines a unified exception hierarchy for the extraction step, allowing the CLI to present typed, user-friendly error messages.

## How to Run the Extraction Phase

### Command-Line Execution

Extract a PDF and write cleaned text to the default output folder:

```bash
python -m scripts.extract path/to/book.pdf --out-dir ./extracted/

```

### Programmatic API Usage

Invoke the CLI logic directly from Python by passing a list of arguments to `main()`:

```python
from book_to_skill.cli import main

args = ["path/to/book.epub", "--out-dir", "out"]
main(args)  # Returns None; output written to disk

```

### Direct Parser Access

Bypass the CLI entirely to work with extracted text in memory:

```python
from book_to_skill.parsers.pdf import PDFParser
from book_to_skill.sanitize import sanitize_text

raw = PDFParser().parse("my_book.pdf")
clean = sanitize_text(raw)
print(clean[:500])  # Preview first 500 characters

```

## Summary

- The EXTRACTOR phase is initiated via [`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py), which ensures UTF-8 I/O before calling the package CLI.
- [`book_to_skill/cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/cli.py) coordinates the workflow, routing files to the correct parser in `book_to_skill/parsers/`.
- Seven format-specific parsers implement the `parse(path: str) -> str` interface for PDF, EPUB, DOCX, HTML, RTF, plain text, and Calibre-compatible formats.
- [`book_to_skill/sanitize.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/sanitize.py) provides the `sanitize_text()` function to normalize Unicode and remove control characters.
- Configuration defaults live in [`book_to_skill/config.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/config.py), while [`book_to_skill/exceptions.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/exceptions.py) supplies the error taxonomy.

## Frequently Asked Questions

### What file formats does the EXTRACTOR phase support?

The EXTRACTOR phase supports PDF, EPUB, Microsoft Word (DOCX), RTF, HTML, plain text, and any format convertible via Calibre. Each format has a dedicated parser module in `book_to_skill/parsers/` that implements the standard `parse()` interface.

### How does the CLI select the correct parser for a document?

The `main()` function in [`book_to_skill/cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/cli.py) inspects the file extension of the input path. It maps extensions to parser classes—for example, `.pdf` routes to `PDFParser`—and instantiates the appropriate class to extract text.

### Can I extract text programmatically without writing to disk?

Yes. Import the specific parser from `book_to_skill.parsers` (such as `PDFParser` or `EPUBParser`) and call its `parse()` method directly. Pass the resulting string to `sanitize_text()` from [`book_to_skill/sanitize.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/sanitize.py) to replicate the full pipeline in memory without filesystem I/O.

### Where is the extracted text sanitized during the pipeline?

Sanitization occurs in [`book_to_skill/sanitize.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/sanitize.py). After the CLI receives raw text from a parser, it passes the string through `sanitize_text()` to normalize Unicode, strip control characters, and remove extraction artifacts before writing the final output.