# What Is the Spec-Driven Generator in book-to-skill?

> Discover the spec-driven generator in book-to-skill. This orchestration engine transforms documents into structured Agent Skills by reading specifications, managing interactions, and handling token budgets.

- Repository: [Virgilio Junior/book-to-skill](https://github.com/virgiliojr94/book-to-skill)
- Tags: deep-dive
- Published: 2026-08-24

---

**The spec-driven generator is the orchestration engine of the book-to-skill pipeline that reads the [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) specification, coordinates multi-step user interactions, manages token budgets, and transforms extracted document content into structured, host-ready Agent Skills.**

The book-to-skill project implements a two-phase architecture that separates deterministic text extraction from intelligent skill generation. While the first phase uses pure Python to extract clean text from PDFs, EPUBs, and DOCX files, the **spec-driven generator** serves as the second phase—an agent-driven workflow system that interprets a declarative specification to produce consumable AI skills.

## The Generator's Role in the Architecture

According to [`docs/architecture.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/docs/architecture.md), the pipeline deliberately splits responsibilities between extraction and generation. The **spec-driven generator** maintains this separation by remaining language-agnostic and driven entirely by the markdown specification in [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md). This design allows the deterministic extractor ([`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py)) to focus solely on document parsing while the generator handles the complex logic of skill composition, user prompting, and output formatting.

The generator operates by treating [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) as an executable blueprint. It executes numbered steps sequentially, collecting user inputs like `BOOK_TYPE` and `DEPTH`, calculating token budgets, and ultimately producing a directory structure containing `chapters/`, [`glossary.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/glossary.md), [`patterns.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/patterns.md), [`cheatsheet.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/cheatsheet.md), and the final [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) that host agents consume.

## Step-by-Step Workflow Orchestration

The generator follows a precise sequence defined in the specification, with each step mapped to specific actions and file operations.

### Step 0–1.5: Content Classification (BOOK_TYPE)

Before processing begins, the generator queries the user to classify the book as either *technical* or *text-heavy*, storing the result as `BOOK_TYPE` in [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) (lines 104–118). This classification directly influences downstream token allocation decisions, ensuring technical manuals receive different processing resources than narrative text.

### Step 2–2.5: Extraction and Data Loading

Using the classified `BOOK_TYPE`, the generator invokes the deterministic extraction layer:

```bash
scripts/extract.py

```

Following extraction, the generator reads the intermediate artefacts [`full_text.txt`](https://github.com/virgiliojr94/book-to-skill/blob/main/full_text.txt) and [`metadata.json`](https://github.com/virgiliojr94/book-to-skill/blob/main/metadata.json) from the working directory. This step bridges the extractor's output with the generator's orchestration logic, ensuring raw content is available for structural analysis.

### Step 3–4: Structural Analysis and Purpose Definition (DEPTH)

The generator analyzes the loaded text to identify the book's title, author, and chapter hierarchy. It then prompts the user to define the skill's intended purpose and depth, capturing this as `DEPTH` (lines 79–95). This parameter controls the granularity of chapter summaries and determines how extensively the generator processes each section.

### Step 6–9: Skill Assembly and File Generation

At this stage, the generator creates the complete file hierarchy required by host agents:

- `chapters/` — Individual chapter summaries and extracts
- [`glossary.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/glossary.md) — Term definitions extracted from context
- [`patterns.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/patterns.md) — Recurring concepts and structures
- [`cheatsheet.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/cheatsheet.md) — Quick reference material

Simultaneously, it composes the master [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) file (lines 56–64) that serves as the entry point for AI agents using the skill.

### Step 9.5: Security Validation

Before finalization, the generator executes mandatory security scanning via [`tools/scan_generated_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/scan_generated_skill.py) (and optionally validates through [`tools/validate_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/validate_skill.py)) to detect hidden instructions, invisible Unicode characters, or prompt injection attempts that may have survived the extraction process (lines 28–33). This step ensures the generated skill contains only the intended instructional content.

### Step 10–11: Cleanup and Publication

The generator removes temporary working directories, reports completion status, and optionally publishes the finished skill to GitHub (lines 41–49). This cleanup phase ensures no intermediate files containing full text extracts persist on the system after skill generation.

## Key Components and Source Files

Several critical files enable the generator's functionality within the `virgiliojr94/book-to-skill` repository:

- **[`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md)** — The specification document itself, containing numbered steps, parameter definitions, and output formats that the generator interprets as instructions.
- **[`docs/architecture.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/docs/architecture.md)** — Documents the two-phase architecture and the generator's position as the orchestration layer distinct from the extraction engine.
- **[`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py)** — The thin shim that launches the deterministic extractor; called by the generator during Step 2–2.5.
- **[`tools/scan_generated_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/scan_generated_skill.py)** — Implements the security auditing logic for Step 9.5, scanning for anomalous content in generated artefacts.
- **[`tools/validate_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/validate_skill.py)** — Provides additional validation capabilities for checking previously generated skills against the specification.

## Practical Usage Examples

### Running the Complete Pipeline

To convert a book and automatically trigger the generator workflow:

```bash
book-to-skill path/to/book.pdf my-awesome-skill

```

This command executes the extractor, then hands control to the spec-driven generator to produce the skill in the appropriate host directory (e.g., `~/.copilot/skills/my-awesome-skill/`).

### Validating a Generated Skill

To run the security scanning step (Step 9.5) independently:

```bash
python -m tools.validate_skill.py SKILL.md

```

This validates that an existing skill complies with security requirements defined in the specification.

### Updating Existing Skills

The generator supports incremental updates through its *Update / Fold-in Workflow*:

```bash
book-to-skill new/chapter1.pdf existing-skill-slug --mode text

```

Here, the generator reads the existing skill's [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md), merges new chapter content, updates glossary entries, and regenerates the master specification without requiring full reprocessing of the original document.

## Summary

- The **spec-driven generator** serves as the orchestration layer that converts extracted text into structured Agent Skills by following the declarative [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) specification.
- It manages **user interactions** to determine `BOOK_TYPE` and `DEPTH`, which control processing budgets and output granularity.
- The generator coordinates with **deterministic extractors** by reading [`full_text.txt`](https://github.com/virgiliojr94/book-to-skill/blob/main/full_text.txt) and [`metadata.json`](https://github.com/virgiliojr94/book-to-skill/blob/main/metadata.json), then produces standardized output directories and files.
- **Security scanning** via [`tools/scan_generated_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/scan_generated_skill.py) ensures generated content is free from hidden instructions before delivery.
- By separating specification from implementation, the generator remains **language-agnostic** while the extractor handles pure Python document processing.

## Frequently Asked Questions

### How does the spec-driven generator differ from the extractor?

The extractor is a deterministic Python component focused solely on parsing PDFs, EPUBs, and DOCX files into clean text ([`full_text.txt`](https://github.com/virgiliojr94/book-to-skill/blob/main/full_text.txt) and [`metadata.json`](https://github.com/virgiliojr94/book-to-skill/blob/main/metadata.json)). The generator is an orchestration engine that reads the [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) specification, makes decisions based on user input, and transforms that extracted text into structured skills. The extractor knows nothing about skill formats or AI agents; the generator knows nothing about PDF parsing internals.

### What is the SKILL.md file?

[`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) is both the specification document and the final output artefact. During generation, it serves as the executable blueprint containing numbered steps (0–11) that the generator follows. Upon completion, it becomes the master file that host agents load to understand the skill's capabilities, chapter structure, and usage patterns.

### How does the generator handle different types of books?

During Step 0–1.5, the generator captures `BOOK_TYPE` (technical versus text-heavy) from the user. This classification influences token budget calculations throughout the pipeline. Technical books typically receive more detailed glossary and pattern extraction, while text-heavy books may prioritize narrative summarization and thematic analysis over structural pattern recognition.

### What security measures does the generator implement?

At Step 9.5, the generator automatically runs [`tools/scan_generated_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/scan_generated_skill.py) to audit all generated files for invisible Unicode characters, potential prompt injection sequences, or hidden instructions that could have been embedded in the source document. This scan occurs before the skill is copied to the final destination directory, ensuring only sanitized content reaches the host agent environment.