Steps in the GENERATOR Phase of Book-to-Skill: From Raw Text to Agent Skills

The GENERATOR phase of Book-to-Skill executes eight deterministic steps (Steps 3 through 10.5) that convert extracted book text into a modular Agent Skill package, including chapter summaries, auxiliary documentation, a master manifest, and optional security validation and publishing.

The GENERATOR phase begins immediately after the extraction process (Step 2) completes, transforming the raw full_text.txt output from scripts/extract.py into a fully-featured Agent Skill. According to the virgiliojr94/book-to-skill repository architecture, this phase follows a specification-driven workflow defined in docs/how-it-works.md, producing structured markdown optimized for host agents like Copilot CLI, Amp, and Claude Code.

Step 3: Analyze Book Structure

The generator reads the first approximately 8,000 characters of full_text.txt to extract critical metadata. As documented in lines 44-56 of docs/how-it-works.md, this step identifies the book title, author, chapter headings, table of contents, and estimates the total chapter count.

If the user invoked the tool with the --analyze-only flag, the workflow terminates here, outputting an extraction report without proceeding to skill generation.

Step 4: Configure Skill Purpose and Depth

In full conversion mode, the generator prompts the user to select the skill’s intended purpose according to lines 78-96 of the specification. Available options include applying frameworks, thinking with mental models, referencing chapters, or a combination thereof.

The user’s selection drives the DEPTH flag, which accepts values of reference or study. This flag determines the granularity of subsequent content generation, particularly affecting the token budget allocation matrix applied in Step 7.

Step 5: Determine Skill Name and Location

This step generates a URL-friendly slug for the new skill, typically based on the author-concept combination or book title. As implemented in lines 99-130, the generator selects the appropriate skill home directory based on the detected host agent environment (Copilot CLI, Amp, Claude Code, etc.).

The configuration mapping for host-specific root directories resides in book_to_skill/config.py, ensuring the skill installs to the correct location for immediate agent consumption.

Step 6: Create Skill Directory Structure

Following the specification in lines 133-138, the generator creates the top-level skill folder and a chapters/ subdirectory under the designated SKILLS_HOME path. This establishes the modular structure required for on-demand loading, where agents fetch only specific chapter files needed for a query rather than loading the entire book text.

Step 7: Generate Chapter Summaries

The core content generation occurs here. For each detected chapter, the generator reads the corresponding slice of full_text.txt and produces a dedicated markdown file (`chNN-<slug>.md`) in the chapters/ directory.

According to lines 140-207 of docs/how-it-works.md, the generator selects a per-chapter token budget from a matrix that considers both BOOK_TYPE (technical versus text) and DEPTH (reference versus study). The resulting template includes:

• Core Idea: Central concept summary
• Frameworks Introduced: Named methodologies with usage conditions
• Code Examples: Language-specific implementations (technical books only)
• Worked Example: Detailed applications (study depth only)

Example output structure from a generated chapter file:

# Chapter 3: Learning to Iterate

## Core Idea

Iterative learning beats upfront perfection; each cycle refines the model.

## Frameworks Introduced

- **Iterative Feedback Loop**: repeat → evaluate → improve
  - When to use: when requirements evolve rapidly.
  - How: run a short experiment, collect metrics, update the design.

## Worked Example

```python
def feedback_loop(model, data):
    for i in range(3):
        model.fit(data)
        loss = evaluate(model, data)
        print(f"Round {i}: loss={loss}")

Key Takeaways

1. Start with a minimal viable prototype.
2. Measure early and often.
3. Embrace failure as a learning signal.

## Step 8: Generate Supporting Files

The generator produces three auxiliary reference documents as specified in lines 220-255:

- **[`glossary.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/glossary.md)**: Alphabetical definitions of key terms with individual token ceilings
- **[`patterns.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/patterns.md)**: Concrete techniques, design patterns, and reusable recipes extracted from the text
- **[`cheatsheet.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/cheatsheet.md)**: Decision rules, threshold tables, and quick-reference criteria

Each auxiliary file adheres to strict token budgets defined in the configuration to ensure optimal retrieval performance by host agents.

## Step 9: Assemble the Master SKILL.md

This step constructs the top-level skill manifest that agents consume first. As documented in lines 258-326, the [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) file assembles:

1. Header metadata and usage instructions
2. Compact **Core Frameworks** section (approximately 2,000 tokens)
3. Chapter Index linking to generated chapter files
4. Topic Index for cross-referencing concepts
5. Links to supporting auxiliary files

The entire manifest must remain under 4,000 tokens; the generator truncates excess content from the end to maintain this constraint.

## Step 9.5: Security Scanning

Before finalizing, the generator executes the security scanner against the newly created skill directory. The tool invokes [`tools/scan_generated_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/scan_generated_skill.py) as described in lines 328-337.

If the scanner exits with a non-zero status, the process halts immediately, requiring the user to review security findings before proceeding. Only clean scans continue to the cleanup phase.

## Step 10: Cleanup and Reporting

The generator deletes the temporary work directory created by the extractor (Step 2), then outputs a concise success report to the terminal. According to lines 340-421, this report includes:

- Absolute path to the generated skill directory
- Total chapter count generated
- Token estimates for the master file and individual chapters
- Example usage commands for the target host agent

## Step 10.5: Optional Publishing

The final optional step, documented in lines 424-467, offers to publish the generated skill to GitHub as either a public or private repository. This step respects copyright and visibility rules, and only executes after a successful security scan. The publishing workflow automates repository creation and pushes the complete skill package to the remote.

## CLI Execution Examples

The generator phase initiates automatically after extraction completes, but supports several invocation patterns:

```bash

# Standard workflow: extract then generate

book-to-skill ./my-book.pdf --mode technical

# Analysis-only mode: stops after Step 3

book-to-skill ./my-book.pdf --mode text --analyze-only

# Skip interactive prompts using defaults

book-to-skill ./my-book.pdf --mode text --no-interactive

Generated skills follow this directory structure under the host-specific skill home:

~/.copilot/skills/design-thinking/
├── SKILL.md
├── chapters/
│   ├── ch01-introduction.md
│   └── ch02-iteration.md
├── glossary.md
├── patterns.md
└── cheatsheet.md

Summary

• The GENERATOR phase processes extracted text through eight deterministic steps (3 through 10.5) to produce a modular Agent Skill ready for host consumption.
• Analysis (Step 3) extracts metadata from the first 8,000 characters and supports an early-exit analysis mode via --analyze-only.
• Configuration (Steps 4-6) establishes the DEPTH flag, generates the skill slug, and creates the directory structure under the host agent’s SKILLS_HOME.
• Content Generation (Steps 7-9) creates chapter files with token-budget-aware templates (based on BOOK_TYPE and DEPTH) and assembles the master SKILL.md manifest under a strict 4,000-token limit.
• Validation (Step 9.5) runs tools/scan_generated_skill.py to enforce security policies before completion.
• Deployment (Steps 10-10.5) cleans temporary files, reports generation metrics, and optionally publishes the skill to GitHub.

Frequently Asked Questions

What triggers the GENERATOR phase to start?

The GENERATOR phase begins automatically after the extractor (Step 2) finishes writing full_text.txt and metadata.json to the temporary work directory. The generator reads these artifacts to begin Step 3 (structure analysis). You cannot run the generator independently without first extracting text from a source document using scripts/extract.py.

How does Book-to-Skill handle token limits during generation?

The generator enforces strict token budgets defined in book_to_skill/config.py. The master SKILL.md truncates content exceeding 4,000 tokens, while chapter files use a matrix-based budget considering both BOOK_TYPE (technical versus text) and DEPTH (reference versus study). Supporting files like glossary.md and patterns.md each have independent ceilings to prevent context overflow in host agents.

Can I skip the security scanning step?

No, the security scan in Step 9.5 is mandatory for the workflow to complete successfully. If tools/scan_generated_skill.py returns a non-zero exit code, the generator stops and requires you to review the findings. However, you can manually inspect the generated skill directory before the scan runs by interrupting the process after Step 9 completes.

Where does the generator place the final skill files?

The generator installs skills to a host-specific skill home directory determined during Step 5. For example, Copilot CLI skills install to ~/.copilot/skills/, while Amp and Claude Code use their respective configuration directories mapped in book_to_skill/config.py. You can override this behavior by manually moving the generated folder after Step 10 completes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}