# What Is the Spec-Driven Generator in book-to-skill? Architecture and Workflow Explained

> Understand the spec-driven generator in book-to-skill. Learn how this pipeline stage transforms raw text into structured Agent Skills using the SKILL.md specification.

- Repository: [Virgilio Junior/book-to-skill](https://github.com/virgiliojr94/book-to-skill)
- Tags: architecture
- Published: 2026-08-26

---

**The spec-driven generator is the second stage of the book-to-skill pipeline that reads the [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) specification and orchestrates agent-driven steps to transform raw extracted text into a structured, on-demand Agent Skill.**

The **spec-driven generator** serves as the orchestration engine in the **virgiliojr94/book-to-skill** repository, bridging the gap between deterministic text extraction and flexible skill generation. While the Python extractor handles document parsing, the generator interprets the declarative workflow defined in [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) to produce structured AI-ready assets. This architecture separates extraction concerns from generation logic, enabling language-agnostic skill creation through a single markdown specification.

## How the Spec-Driven Generator Fits Into the Pipeline

The book-to-skill pipeline operates as a two-stage system. The first half consists of a **deterministic Python extractor** that converts PDF, EPUB, DOCX, or HTML sources into clean text and metadata. The **spec-driven generator** constitutes the second half, consuming the extracted data and the formal specification to produce the final skill artifacts.

This separation ensures that the extractor remains pure Python while the generator stays flexible and language-agnostic. The generator reads the numbered steps, token budgets, and output formats defined in [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) to coordinate user interactions and file generation.

## Step-by-Step Workflow of the Spec-Driven Generator

The generator follows a precise sequence defined in the [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) specification, ranging from initial content classification through final security validation.

### Content Type Classification (Steps 0–1.5)

Before processing begins, the generator queries the user for the **content type** (technical versus text-heavy) and stores this value as **`BOOK_TYPE`**. This classification directly influences later token budget decisions and processing depth. This interaction is defined in [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) within the *Identify content type* section (lines 104–118).

### Text Extraction Phase (Steps 2–2.5)

Using the established `BOOK_TYPE`, the generator launches **[`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py)** to handle the deterministic extraction. Upon completion, the generator reads the resulting **[`full_text.txt`](https://github.com/virgiliojr94/book-to-skill/blob/main/full_text.txt)** and **[`metadata.json`](https://github.com/virgiliojr94/book-to-skill/blob/main/metadata.json)** files to prepare for structural analysis. This phase is documented in the *Extract text from the source documents* section of the specification.

### Structural Analysis and Depth Configuration (Steps 3–4)

The generator analyzes the extracted text to discover the book's title, author, chapter structure, and core themes. During this phase, it prompts the user to define **`DEPTH`**—the specific purpose and detail level the skill should provide. This parameter drives how thoroughly each chapter summary gets processed, as implemented in the *Analyze book structure* and *Ask purpose* sections (lines 44–53 and 79–95).

### Skill Asset Generation (Steps 6–9)

Based on the analysis and depth parameters, the generator creates the on-demand file hierarchy:

- **`chapters/`** directory with processed content
- **[`glossary.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/glossary.md)** for terminology
- **[`patterns.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/patterns.md)** for reusable concepts
- **[`cheatsheet.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/cheatsheet.md)** for quick reference
- Master **[`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md)** for host agent consumption

These outputs are dictated by the *Create skill directory structure* section (lines 33–41 and 56–64) of the specification.

### Security Validation (Step 9.5)

Before finalizing the skill, the generator executes **[`tools/scan_generated_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/scan_generated_skill.py)** to perform a security scan. This step checks for hidden instructions, invisible Unicode characters, or other potentially malicious content that may have leaked into the generated files. The procedure is defined in the *Scan the generated skill* section (lines 28–33).

### Finalization and Publication (Steps 10–11)

The generator cleans up the temporary work directory, reports the completed skill status, and optionally publishes the artifact to GitHub. These final operations follow the *Cleanup and report* and *Publish* sections (lines 41–49 and 14–22).

## Key Files Driving the Spec-Driven Generator

Several critical files enable the generator's operation:

- **[`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md)** — The core specification document containing the step-by-step workflow, token budgets, and output formats that the generator interprets.
- **[`docs/architecture.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/docs/architecture.md)** — Documentation describing the two-half architecture and the generator's role in the pipeline.
- **[`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py)** — The thin shim that launches the deterministic extractor; called by the generator during Step 2.
- **[`tools/validate_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/validate_skill.py)** and **[`tools/scan_generated_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/scan_generated_skill.py)** — Security utilities implementing Step 9.5's validation requirements.

## Practical Usage Examples

The spec-driven generator supports multiple operational modes through the `book-to-skill` CLI:

### Full Conversion Workflow

```bash
book-to-skill path/to/book.pdf my-awesome-skill

```

This command automatically triggers [`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py) and follows the complete [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) workflow, producing a skill under the appropriate host-specific directory (e.g., `~/.copilot/skills/my-awesome-skill/`).

### Security Validation Only

```bash
python -m tools.validate_skill.py SKILL.md

```

This invokes the validation step (Step 9.5) defined in the spec to check a previously generated skill for security concerns without re-running the full pipeline.

### Incremental Updates

```bash
book-to-skill new/chapter1.pdf existing-skill-slug --mode text

```

The generator reads the existing skill's [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md), merges new chapter summaries and glossary entries, and updates the master specification according to the *Update / Fold-in Workflow* section.

## Summary

- The **spec-driven generator** serves as the orchestration layer that converts extracted text into structured Agent Skills by following the declarative [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) specification.
- It operates as the second half of a two-stage pipeline, remaining language-agnostic while the deterministic extractor handles pure Python parsing.
- The generator manages **user-defined parameters** (`BOOK_TYPE` and `DEPTH`) that control token budgets and content detail levels.
- It produces a standardized file hierarchy including `chapters/`, [`glossary.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/glossary.md), [`patterns.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/patterns.md), and the master [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) for host agents.
- **Security scanning** via [`tools/scan_generated_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/scan_generated_skill.py) is mandatory before finalization to prevent hidden instruction injection.

## Frequently Asked Questions

### What makes the generator "spec-driven"?

The generator is "spec-driven" because it does not contain hardcoded business logic for skill creation. Instead, it reads and executes the numbered steps, constraints, and output formats defined in the **[`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md)** markdown file. This allows the same generator code to produce vastly different skill structures simply by modifying the specification, making the system adaptable to new AI host requirements without code changes.

### How does the generator handle different document types?

The generator relies on the **`BOOK_TYPE`** variable collected in Steps 0–1.5 to adjust its processing strategy. Technical books receive different token allocations and structural parsing compared to text-heavy narrative works. This classification influences how the generator prepares the extracted content before creating chapter summaries and reference materials.

### What security measures prevent malicious content in generated skills?

Before completion, the generator mandatory runs **[`tools/scan_generated_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/scan_generated_skill.py)** (Step 9.5) to detect hidden Unicode characters, invisible instructions, or prompt injection attempts that might have survived the extraction phase. This security gate ensures that only sanitized content enters the final skill artifacts loaded by host agents.

### Can the generator update existing skills rather than creating new ones?

Yes, the generator supports an *Update / Fold-in Workflow* that reads an existing skill's [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md), extracts the current structure, and merges new content from additional source files. This incremental mode preserves existing user edits while incorporating new chapters or updated metadata, triggered via the `--mode text` flag or similar update commands.