What Is the Spec-Driven Generator in book-to-skill? Architecture and Workflow Explained

The spec-driven generator is the second stage of the book-to-skill pipeline that reads the SKILL.md specification and orchestrates agent-driven steps to transform raw extracted text into a structured, on-demand Agent Skill.

The spec-driven generator serves as the orchestration engine in the virgiliojr94/book-to-skill repository, bridging the gap between deterministic text extraction and flexible skill generation. While the Python extractor handles document parsing, the generator interprets the declarative workflow defined in SKILL.md to produce structured AI-ready assets. This architecture separates extraction concerns from generation logic, enabling language-agnostic skill creation through a single markdown specification.

How the Spec-Driven Generator Fits Into the Pipeline

The book-to-skill pipeline operates as a two-stage system. The first half consists of a deterministic Python extractor that converts PDF, EPUB, DOCX, or HTML sources into clean text and metadata. The spec-driven generator constitutes the second half, consuming the extracted data and the formal specification to produce the final skill artifacts.

This separation ensures that the extractor remains pure Python while the generator stays flexible and language-agnostic. The generator reads the numbered steps, token budgets, and output formats defined in SKILL.md to coordinate user interactions and file generation.

Step-by-Step Workflow of the Spec-Driven Generator

The generator follows a precise sequence defined in the SKILL.md specification, ranging from initial content classification through final security validation.

Content Type Classification (Steps 0–1.5)

Before processing begins, the generator queries the user for the content type (technical versus text-heavy) and stores this value as BOOK_TYPE. This classification directly influences later token budget decisions and processing depth. This interaction is defined in SKILL.md within the Identify content type section (lines 104–118).

Text Extraction Phase (Steps 2–2.5)

Using the established BOOK_TYPE, the generator launches scripts/extract.py to handle the deterministic extraction. Upon completion, the generator reads the resulting full_text.txt and metadata.json files to prepare for structural analysis. This phase is documented in the Extract text from the source documents section of the specification.

Structural Analysis and Depth Configuration (Steps 3–4)

The generator analyzes the extracted text to discover the book's title, author, chapter structure, and core themes. During this phase, it prompts the user to define DEPTH—the specific purpose and detail level the skill should provide. This parameter drives how thoroughly each chapter summary gets processed, as implemented in the Analyze book structure and Ask purpose sections (lines 44–53 and 79–95).

Skill Asset Generation (Steps 6–9)

Based on the analysis and depth parameters, the generator creates the on-demand file hierarchy:

• chapters/ directory with processed content
• glossary.md for terminology
• patterns.md for reusable concepts
• cheatsheet.md for quick reference
• Master SKILL.md for host agent consumption

These outputs are dictated by the Create skill directory structure section (lines 33–41 and 56–64) of the specification.

Security Validation (Step 9.5)

Before finalizing the skill, the generator executes tools/scan_generated_skill.py to perform a security scan. This step checks for hidden instructions, invisible Unicode characters, or other potentially malicious content that may have leaked into the generated files. The procedure is defined in the Scan the generated skill section (lines 28–33).

Finalization and Publication (Steps 10–11)

The generator cleans up the temporary work directory, reports the completed skill status, and optionally publishes the artifact to GitHub. These final operations follow the Cleanup and report and Publish sections (lines 41–49 and 14–22).

Key Files Driving the Spec-Driven Generator

Several critical files enable the generator's operation:

• SKILL.md — The core specification document containing the step-by-step workflow, token budgets, and output formats that the generator interprets.
• docs/architecture.md — Documentation describing the two-half architecture and the generator's role in the pipeline.
• scripts/extract.py — The thin shim that launches the deterministic extractor; called by the generator during Step 2.
• tools/validate_skill.py and tools/scan_generated_skill.py — Security utilities implementing Step 9.5's validation requirements.

Practical Usage Examples

The spec-driven generator supports multiple operational modes through the book-to-skill CLI:

Full Conversion Workflow

book-to-skill path/to/book.pdf my-awesome-skill

This command automatically triggers scripts/extract.py and follows the complete SKILL.md workflow, producing a skill under the appropriate host-specific directory (e.g., ~/.copilot/skills/my-awesome-skill/).

Security Validation Only

python -m tools.validate_skill.py SKILL.md

This invokes the validation step (Step 9.5) defined in the spec to check a previously generated skill for security concerns without re-running the full pipeline.

Incremental Updates

book-to-skill new/chapter1.pdf existing-skill-slug --mode text

The generator reads the existing skill's SKILL.md, merges new chapter summaries and glossary entries, and updates the master specification according to the Update / Fold-in Workflow section.

Summary

• The spec-driven generator serves as the orchestration layer that converts extracted text into structured Agent Skills by following the declarative SKILL.md specification.
• It operates as the second half of a two-stage pipeline, remaining language-agnostic while the deterministic extractor handles pure Python parsing.
• The generator manages user-defined parameters (BOOK_TYPE and DEPTH) that control token budgets and content detail levels.
• It produces a standardized file hierarchy including chapters/, glossary.md, patterns.md, and the master SKILL.md for host agents.
• Security scanning via tools/scan_generated_skill.py is mandatory before finalization to prevent hidden instruction injection.

Frequently Asked Questions

What makes the generator "spec-driven"?

The generator is "spec-driven" because it does not contain hardcoded business logic for skill creation. Instead, it reads and executes the numbered steps, constraints, and output formats defined in the SKILL.md markdown file. This allows the same generator code to produce vastly different skill structures simply by modifying the specification, making the system adaptable to new AI host requirements without code changes.

How does the generator handle different document types?

The generator relies on the BOOK_TYPE variable collected in Steps 0–1.5 to adjust its processing strategy. Technical books receive different token allocations and structural parsing compared to text-heavy narrative works. This classification influences how the generator prepares the extracted content before creating chapter summaries and reference materials.

What security measures prevent malicious content in generated skills?

Before completion, the generator mandatory runs tools/scan_generated_skill.py (Step 9.5) to detect hidden Unicode characters, invisible instructions, or prompt injection attempts that might have survived the extraction phase. This security gate ensures that only sanitized content enters the final skill artifacts loaded by host agents.

Can the generator update existing skills rather than creating new ones?

Yes, the generator supports an Update / Fold-in Workflow that reads an existing skill's SKILL.md, extracts the current structure, and merges new content from additional source files. This incremental mode preserves existing user edits while incorporating new chapters or updated metadata, triggered via the --mode text flag or similar update commands.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}