What Is the Spec-Driven Generator in book-to-skill?

The spec-driven generator is the orchestration engine of the book-to-skill pipeline that reads the SKILL.md specification, coordinates multi-step user interactions, manages token budgets, and transforms extracted document content into structured, host-ready Agent Skills.

The book-to-skill project implements a two-phase architecture that separates deterministic text extraction from intelligent skill generation. While the first phase uses pure Python to extract clean text from PDFs, EPUBs, and DOCX files, the spec-driven generator serves as the second phase—an agent-driven workflow system that interprets a declarative specification to produce consumable AI skills.

The Generator's Role in the Architecture

According to docs/architecture.md, the pipeline deliberately splits responsibilities between extraction and generation. The spec-driven generator maintains this separation by remaining language-agnostic and driven entirely by the markdown specification in SKILL.md. This design allows the deterministic extractor (scripts/extract.py) to focus solely on document parsing while the generator handles the complex logic of skill composition, user prompting, and output formatting.

The generator operates by treating SKILL.md as an executable blueprint. It executes numbered steps sequentially, collecting user inputs like BOOK_TYPE and DEPTH, calculating token budgets, and ultimately producing a directory structure containing chapters/, glossary.md, patterns.md, cheatsheet.md, and the final SKILL.md that host agents consume.

Step-by-Step Workflow Orchestration

The generator follows a precise sequence defined in the specification, with each step mapped to specific actions and file operations.

Step 0–1.5: Content Classification (BOOK_TYPE)

Before processing begins, the generator queries the user to classify the book as either technical or text-heavy, storing the result as BOOK_TYPE in SKILL.md (lines 104–118). This classification directly influences downstream token allocation decisions, ensuring technical manuals receive different processing resources than narrative text.

Step 2–2.5: Extraction and Data Loading

Using the classified BOOK_TYPE, the generator invokes the deterministic extraction layer:

scripts/extract.py

Following extraction, the generator reads the intermediate artefacts full_text.txt and metadata.json from the working directory. This step bridges the extractor's output with the generator's orchestration logic, ensuring raw content is available for structural analysis.

Step 3–4: Structural Analysis and Purpose Definition (DEPTH)

The generator analyzes the loaded text to identify the book's title, author, and chapter hierarchy. It then prompts the user to define the skill's intended purpose and depth, capturing this as DEPTH (lines 79–95). This parameter controls the granularity of chapter summaries and determines how extensively the generator processes each section.

Step 6–9: Skill Assembly and File Generation

At this stage, the generator creates the complete file hierarchy required by host agents:

• chapters/ — Individual chapter summaries and extracts
• glossary.md — Term definitions extracted from context
• patterns.md — Recurring concepts and structures
• cheatsheet.md — Quick reference material

Simultaneously, it composes the master SKILL.md file (lines 56–64) that serves as the entry point for AI agents using the skill.

Step 9.5: Security Validation

Before finalization, the generator executes mandatory security scanning via tools/scan_generated_skill.py (and optionally validates through tools/validate_skill.py) to detect hidden instructions, invisible Unicode characters, or prompt injection attempts that may have survived the extraction process (lines 28–33). This step ensures the generated skill contains only the intended instructional content.

Step 10–11: Cleanup and Publication

The generator removes temporary working directories, reports completion status, and optionally publishes the finished skill to GitHub (lines 41–49). This cleanup phase ensures no intermediate files containing full text extracts persist on the system after skill generation.

Key Components and Source Files

Several critical files enable the generator's functionality within the virgiliojr94/book-to-skill repository:

• SKILL.md — The specification document itself, containing numbered steps, parameter definitions, and output formats that the generator interprets as instructions.
• docs/architecture.md — Documents the two-phase architecture and the generator's position as the orchestration layer distinct from the extraction engine.
• scripts/extract.py — The thin shim that launches the deterministic extractor; called by the generator during Step 2–2.5.
• tools/scan_generated_skill.py — Implements the security auditing logic for Step 9.5, scanning for anomalous content in generated artefacts.
• tools/validate_skill.py — Provides additional validation capabilities for checking previously generated skills against the specification.

Practical Usage Examples

Running the Complete Pipeline

To convert a book and automatically trigger the generator workflow:

book-to-skill path/to/book.pdf my-awesome-skill

This command executes the extractor, then hands control to the spec-driven generator to produce the skill in the appropriate host directory (e.g., ~/.copilot/skills/my-awesome-skill/).

Validating a Generated Skill

To run the security scanning step (Step 9.5) independently:

python -m tools.validate_skill.py SKILL.md

This validates that an existing skill complies with security requirements defined in the specification.

Updating Existing Skills

The generator supports incremental updates through its Update / Fold-in Workflow:

book-to-skill new/chapter1.pdf existing-skill-slug --mode text

Here, the generator reads the existing skill's SKILL.md, merges new chapter content, updates glossary entries, and regenerates the master specification without requiring full reprocessing of the original document.

Summary

• The spec-driven generator serves as the orchestration layer that converts extracted text into structured Agent Skills by following the declarative SKILL.md specification.
• It manages user interactions to determine BOOK_TYPE and DEPTH, which control processing budgets and output granularity.
• The generator coordinates with deterministic extractors by reading full_text.txt and metadata.json, then produces standardized output directories and files.
• Security scanning via tools/scan_generated_skill.py ensures generated content is free from hidden instructions before delivery.
• By separating specification from implementation, the generator remains language-agnostic while the extractor handles pure Python document processing.

Frequently Asked Questions

How does the spec-driven generator differ from the extractor?

The extractor is a deterministic Python component focused solely on parsing PDFs, EPUBs, and DOCX files into clean text (full_text.txt and metadata.json). The generator is an orchestration engine that reads the SKILL.md specification, makes decisions based on user input, and transforms that extracted text into structured skills. The extractor knows nothing about skill formats or AI agents; the generator knows nothing about PDF parsing internals.

What is the SKILL.md file?

SKILL.md is both the specification document and the final output artefact. During generation, it serves as the executable blueprint containing numbered steps (0–11) that the generator follows. Upon completion, it becomes the master file that host agents load to understand the skill's capabilities, chapter structure, and usage patterns.

How does the generator handle different types of books?

During Step 0–1.5, the generator captures BOOK_TYPE (technical versus text-heavy) from the user. This classification influences token budget calculations throughout the pipeline. Technical books typically receive more detailed glossary and pattern extraction, while text-heavy books may prioritize narrative summarization and thematic analysis over structural pattern recognition.

What security measures does the generator implement?

At Step 9.5, the generator automatically runs tools/scan_generated_skill.py to audit all generated files for invisible Unicode characters, potential prompt injection sequences, or hidden instructions that could have been embedded in the source document. This scan occurs before the skill is copied to the final destination directory, ensuring only sanitized content reaches the host agent environment.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}