# Where to Find book-to-skill Pipeline Documentation: Complete Guide

> Find book-to-skill pipeline documentation easily. Explore architecture, how it works, and performance details in the official markdown files within the virgiliojr94/book-to-skill repository.

- Repository: [Virgilio Junior/book-to-skill](https://github.com/virgiliojr94/book-to-skill)
- Tags: api-reference
- Published: 2026-08-26

---

**The official book-to-skill pipeline documentation is located in three core markdown files within the repository's `docs/` directory: [`architecture.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/architecture.md) for system design, [`how-it-works.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/how-it-works.md) for execution workflows, and [`performance.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/performance.md) for benchmarking metrics.**

The book-to-skill pipeline converts books into AI-ready skills through a deterministic extraction phase followed by an agent-driven generation phase. This guide maps the complete documentation structure for the virgiliojr94/book-to-skill repository, providing direct paths to architectural diagrams, source code references, and practical usage examples.

## Core Documentation Files

The project maintains three primary documentation files that collectively explain the pipeline's operation:

- **[`docs/architecture.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/docs/architecture.md)** – Contains the high-level pipeline diagram, component responsibilities, design principles, security hardening measures, and extension points. This file defines how the deterministic extractor and spec-driven generator interact.

- **[`docs/how-it-works.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/docs/how-it-works.md)** – Provides a step-by-step walkthrough of the extractor (handling PDF, EPUB, DOCX, HTML, and RTF formats) and the generator (using Claude-driven skill creation). Includes benchmark tables and design rationale.

- **[`docs/performance.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/docs/performance.md)** – Details benchmarking of extraction methods (comparing pdftotext versus Docling) and provides cost estimates for skill generation.

## Pipeline Architecture Overview

According to the source code in [`docs/architecture.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/docs/architecture.md), the pipeline operates through two distinct stages:

### The Extractor (Deterministic Python)

The extractor is invoked via [`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py), which serves as a thin shim that forwards arguments to `book_to_skill.cli` (implemented in [`book_to_skill/cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/cli.py)). As documented in lines 12-24 of the architecture file, this stage parses each source format through dedicated modules in `book_to_skill/parsers/` and produces a temporary working directory containing:

- [`full_text.txt`](https://github.com/virgiliojr94/book-to-skill/blob/main/full_text.txt) – The extracted text content
- [`metadata.json`](https://github.com/virgiliojr94/book-to-skill/blob/main/metadata.json) – Structured metadata about the source

The parser modules handle specific formats including PDF, EPUB, DOCX, HTML, RTF, Calibre, and plain text. Security hardening is implemented in [`book_to_skill/sanitize.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/sanitize.py), which strips invisible Unicode characters and protects against XXE attacks.

### The Generator (Spec-Driven Agent)

The generator stage reads the extracted text and analyzes its structure according to the specification defined in [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) (located at the repository root). As described in lines 27-38 of [`docs/architecture.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/docs/architecture.md), this Claude-driven agent follows the steps outlined in the specification to produce:

- Per-chapter summaries
- A comprehensive glossary
- Pattern definitions
- A compact [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) file

### Output Locations

The final generated skill is written to host-specific folders as detailed in lines 40-45 of the architecture documentation. Default locations include:

- `~/.copilot/skills/` – For GitHub Copilot integration
- `~/.agents/skills/` – For general AI agent frameworks
- `~/.claude/skills/` – For Claude-specific implementations

## Key Source Files and Entry Points

Understanding the pipeline requires familiarity with these critical files:

| File | Role |
|------|------|
| [`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py) | Entry-point shim that forwards command-line arguments to the CLI extractor |
| [`book_to_skill/cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/cli.py) | Core CLI implementation that parses arguments and orchestrates multi-source extraction |
| `book_to_skill/parsers/` | Directory containing one module per format (`pdf`, `epub`, `docx`, `html`, `rtf`, `calibre`, `text`) handling the actual conversion logic |
| [`book_to_skill/sanitize.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/sanitize.py) | Security module that sanitizes input by removing invisible Unicode characters and preventing XXE attacks |
| [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) | Specification document that drives the agent generator, defining steps, indexes, and output structure |
| [`tools/validate_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/validate_skill.py) | CLI utility that validates generated [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) files against host-specific rules |

## Running the Pipeline

The documentation references several standard command patterns for executing the pipeline. All commands use repository-relative paths; the scripts resolve absolute paths internally.

```bash

# Basic extraction of a PDF in technical mode with skill generation

scripts/extract.py my_book.pdf --mode technical

# Extraction of multiple sources with automatic mode detection

scripts/extract.py chapter1.pdf chapter2.epub

# Run extractor only (no skill generation) for debugging purposes

python -m book_to_skill.cli --extract-only path/to/file.pdf

# Validate a generated SKILL.md against host standards

python tools/validate_skill.py SKILL.md --lens claude

```

## Summary

- **Three core documents** comprise the official book-to-skill pipeline documentation: [`architecture.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/architecture.md) (design), [`how-it-works.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/how-it-works.md) (execution), and [`performance.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/performance.md) (benchmarks).
- **Two-stage architecture**: A deterministic Python extractor (`book_to_skill/parsers/`) followed by a Claude-driven generator guided by [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md).
- **Entry points**: Use [`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py) for full pipeline execution or `python -m book_to_skill.cli` for extraction-only workflows.
- **Security**: Input sanitization occurs in [`book_to_skill/sanitize.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/sanitize.py) to prevent XXE attacks and remove invisible Unicode characters.
- **Validation**: Use [`tools/validate_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/validate_skill.py) to verify generated skills against host-specific requirements before deployment.

## Frequently Asked Questions

### Where is the book-to-skill pipeline architecture explained?

The complete architecture is documented in [`/docs/architecture.md`](https://github.com/virgiliojr94/book-to-skill/blob/main//docs/architecture.md) at the repository root. This file contains the high-level pipeline diagram, explains the separation between the deterministic extractor and spec-driven generator, and details security hardening measures. Lines 12-45 specifically cover the extractor workflow, generator logic, and output directory structures.

### How do I run the book-to-skill pipeline without generating a skill?

To run only the extraction phase without invoking the skill generator, use the `--extract-only` flag with the CLI module: `python -m book_to_skill.cli --extract-only path/to/file.pdf`. This creates the temporary working directory with [`full_text.txt`](https://github.com/virgiliojr94/book-to-skill/blob/main/full_text.txt) and [`metadata.json`](https://github.com/virgiliojr94/book-to-skill/blob/main/metadata.json) without proceeding to the Claude-driven generation stage, making it useful for debugging parser behavior or verifying text extraction quality.

### What determines where the generated skill files are saved?

Output locations are host-specific and configured in [`docs/architecture.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/docs/architecture.md) (lines 40-45). The generator automatically writes skills to framework-specific directories: `~/.copilot/skills/` for GitHub Copilot, `~/.agents/skills/` for general agent frameworks, or `~/.claude/skills/` for Claude implementations. The [`tools/validate_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/validate_skill.py) utility can verify that generated files meet each host's structural requirements.

### Which file formats does the book-to-skill extractor support?

The extractor supports PDF, EPUB, DOCX, HTML, and RTF through dedicated parser modules located in `book_to_skill/parsers/`. Each format has its own implementation file (e.g., [`pdf.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/pdf.py), [`epub.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/epub.py)) that handles format-specific conversion to the standard [`full_text.txt`](https://github.com/virgiliojr94/book-to-skill/blob/main/full_text.txt) and [`metadata.json`](https://github.com/virgiliojr94/book-to-skill/blob/main/metadata.json) interim format used by the generator stage.