# How to Use the extract.py Script in book-to-skill

> Learn to use the extract.py script in book-to-skill to convert PDFs EPUBs DOCX and more into clean text and structured metadata. Start your document conversion pipeline today.

- Repository: [Virgilio Junior/book-to-skill](https://github.com/virgiliojr94/book-to-skill)
- Tags: how-to-guide
- Published: 2026-08-26

---

**The [`extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/extract.py) script is a lightweight entry-point wrapper that adjusts the Python path and delegates execution to the `main()` function in [`book_to_skill/cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/cli.py), initiating a full extraction pipeline that converts PDFs, EPUBs, DOCX files, and other documents into sanitized text and structured metadata.**

The `book-to-skill` repository provides a document extraction toolkit designed to transform books and technical manuals into AI-ready formats. The [`extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/extract.py) script serves as the primary interface to this system, handling everything from dependency resolution to structural analysis while outputting standardized text files and JSON metadata.

## What extract.py Actually Does

[`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py) is intentionally minimal. When executed, it modifies `sys.path` to include the project root directory, ensuring the `book_to_skill` package can be imported correctly. It then calls the `main()` function defined in [`book_to_skill/cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/cli.py)【⁴⁰⁾⁴¹⁾】, transferring control to the core extraction logic.

This architectural separation keeps the entry point thin and maintainable while the heavy lifting occurs within the package modules.

## Running the Extractor: Command-Line Examples

You can invoke the extraction pipeline using any of three equivalent methods:

```bash
./scripts/extract.py <path-or-glob> [options]
python3 -m book_to_skill <path-or-glob> [options]
python3 scripts/extract.py <path-or-glob> [options]

```

### Basic Single-File Extraction

To extract a single document, provide the file path directly:

```bash
./scripts/extract.py mybook.pdf

```

### Batch Processing with Glob Patterns

Process multiple files simultaneously using shell glob patterns:

```bash
./scripts/extract.py books/*.epub

```

### Technical Mode for Complex Layouts

For PDFs with complex layouts, use the technical mode which leverages Docling for layout-aware extraction:

```bash
./scripts/extract.py report.pdf --mode technical

```

### Handling Missing Dependencies

Automatically detect and install missing extractor dependencies with interactive confirmation:

```bash
./scripts/extract.py thesis.docx --install-missing ask

```

To view all available options:

```bash
./scripts/extract.py --help

```

## Inside the Extraction Pipeline

The `main()` function in [`book_to_skill/cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/cli.py) orchestrates a five-stage pipeline that processes documents systematically.

### Argument Parsing and Path Resolution

First, `main()` calls `parse_arguments()` from [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py)【⁴³⁾】 to handle CLI flags like `--mode` and `--install-missing`. It then uses `resolve_input_files()`【⁴⁴⁾】 to expand file paths, directories, or glob patterns into a concrete list of target documents. The utility also calls `prepare_output_dir()`【⁴²⁾】 to ensure the `OUTPUT_DIR` exists before processing begins.

### Document Processing and Text Extraction

For each resolved file, `main()` invokes `extract_single_file()`, which performs format detection and selects the appropriate parser from `book_to_skill/parsers/` (e.g., [`pdf.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/pdf.py), [`epub.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/epub.py)). If `--install-missing` is specified, the system calls `prepare_dependencies()` from [`book_to_skill/dependencies.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/dependencies.py)【⁴⁵⁾】 to fetch required libraries before extraction proceeds.

The extraction engine handles multiple formats including PDF, EPUB, DOCX, TXT, HTML, and RTF, automatically selecting the best backend for each (such as `pdfminer`, `pdftotext`, or `docling` for PDFs).

### Sanitization and Metadata Generation

After raw extraction, the pipeline passes text through `sanitize_extracted_text()` to clean encoding issues and formatting artifacts. It then runs `estimate_tokens()` to calculate token counts and `detect_structure()` to identify chapter boundaries and table-of-contents presence【⁴⁰⁾】.

All results are written to two files defined in [`book_to_skill/config.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/config.py):
- [`full_text.txt`](https://github.com/virgiliojr94/book-to-skill/blob/main/full_text.txt) (`OUTPUT_TEXT`) containing the sanitized document text
- [`metadata.json`](https://github.com/virgiliojr94/book-to-skill/blob/main/metadata.json) (`OUTPUT_META`) containing token counts, page numbers, chapter counts, and structure detection results

The script concludes by printing a summary showing pages processed, word counts, estimated tokens, detected chapters, and TOC detection status.

## Key Source Files in the Architecture

Understanding the codebase structure helps with customization and debugging:

- **[`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py)** – The wrapper script that adjusts `sys.path` and launches `main()`
- **[`book_to_skill/cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/cli.py)** – Implements the CLI interface and the main extraction loop
- **[`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py)** – Contains `parse_arguments()`, `resolve_input_files()`, `extract_single_file()`, `sanitize_extracted_text()`, `estimate_tokens()`, and `detect_structure()`
- **`book_to_skill/parsers/`** – Directory housing format-specific extractors for PDF, EPUB, DOCX, HTML, RTF, and other formats
- **[`book_to_skill/dependencies.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/dependencies.py)** – Manages optional dependency checking and installation via `prepare_dependencies()`
- **[`book_to_skill/config.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/config.py)** – Defines constants including `OUTPUT_DIR`, `OUTPUT_TEXT`, and `OUTPUT_META`

## Summary

- **[`extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/extract.py) is a shim** that bootstraps the environment and calls `book_to_skill/cli.py:main()` to start processing.
- **Three invocation methods** work identically: direct execution, module execution (`-m`), or explicit `python3` calls.
- **Multi-format support** handles PDF, EPUB, DOCX, TXT, HTML, and RTF through specialized parsers in `book_to_skill/parsers/`.
- **Dual output** generates [`full_text.txt`](https://github.com/virgiliojr94/book-to-skill/blob/main/full_text.txt) and [`metadata.json`](https://github.com/virgiliojr94/book-to-skill/blob/main/metadata.json) in the configured `OUTPUT_DIR`.
- **Layout-aware extraction** is available via `--mode technical` using the Docling backend.
- **Dependency management** can be automated with `--install-missing ask` or `yes`.

## Frequently Asked Questions

### Can I run book-to-skill without using extract.py?

Yes. Because [`extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/extract.py) is just a path-adjusting wrapper, you can invoke the package directly as a module using `python3 -m book_to_skill <path> [options]`. This executes the same `main()` function without requiring the shell script wrapper.

### What file formats does extract.py support?

According to the source analysis, the system supports PDF, EPUB, DOCX, TXT, HTML, and RTF files. Each format has a dedicated parser in the `book_to_skill/parsers/` directory, with PDFs supporting multiple extraction backends including `pdfminer`, `pdftotext`, and `docling`.

### Where does extract.py save the extracted content?

Output files are written to the directory specified by `OUTPUT_DIR` in [`book_to_skill/config.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/config.py)【⁴⁶⁾】. By default, the system generates [`full_text.txt`](https://github.com/virgiliojr94/book-to-skill/blob/main/full_text.txt) (the sanitized document content) and [`metadata.json`](https://github.com/virgiliojr94/book-to-skill/blob/main/metadata.json) (containing token estimates, page counts, chapter detection, and TOC status). The script also prints a console summary of the extraction results.

### How does the technical mode differ from standard extraction?

Standard extraction extracts raw text sequentially, while `--mode technical` activates layout-aware processing using Docling. This mode preserves structural information like columns, tables, and formatting hierarchies, making it ideal for complex technical documents where positional context matters.