Generating Skills from Existing Analysis Notes with book-to-skill

The book-to-skill library converts raw analysis notes into structured Agent Skills through a deterministic pipeline that detects existing skill folders, sanitizes input text, extracts semantic frameworks, and incrementally merges new content without overwriting custom metadata.

The virgiliojr94/book-to-skill repository provides a deterministic extraction pipeline designed specifically for generating skills from existing analysis notes and folding them into reusable AI agent capabilities. When you point the tool at a folder containing previous outputs, it activates an incremental update mode that preserves your existing SKILL.md structure while appending new insights.

How the Fold-In Pipeline Works

Input Detection and Mode Selection

When you invoke the CLI with a target path that contains a SKILL.md file and the expected chapter-*.md directory structure, scripts/extract.py automatically triggers fold-in mode (internally designated as mode 4). This detection mechanism distinguishes between creating a fresh skill versus updating an existing one, allowing the pipeline to route processing accordingly.

File Gathering and Type Resolution

The extraction process begins with scripts/extract.py walking the supplied input paths and reading each file with normalized line endings. The helper module book_to_skill/utils.py inspects file signatures to determine document types—supporting PDF, EPUB, DOCX, TXT, and Markdown—and dispatches to the appropriate parser in book_to_skill/parsers/. For plain-text analysis notes, the system invokes book_to_skill/parsers/text.py to handle framework identification.

Sanitization and Preprocessing

Raw text flows through book_to_skill/sanitize.py before semantic analysis begins. This module removes hazardous Unicode control characters, normalizes whitespace patterns, and strips residual HTML or RTF artifacts. The sanitization layer guarantees that downstream parsers receive clean, valid UTF-8 text, preventing parsing errors when processing notes exported from various note-taking applications.

Semantic Extraction of Frameworks

The cleaned text undergoes analysis for named frameworks, decision rules, anti-patterns, and mental models within the parser logic (primarily book_to_skill/parsers/text.py for text files). The extraction engine generates a structured metadata.json object recording chapter boundaries, discovered frameworks, depth-budget allocations, and precise source markers for traceability.

Skill Assembly and Merging

Following extraction, the pipeline populates a temporary working directory at /tmp/book_skill_work/ with three key artifacts:

• full_text.txt – A concatenated view of all sources with embedded source markers
• metadata.json – Aggregated statistics and framework inventories
• chapter-<n>.md files – Individual chapter documents prefixed with skill-headers listing discovered frameworks

The assembler then merges this output with your existing skill folder: new chapters are written as fresh files, the existing SKILL.md descriptor receives updated metadata without overwriting user-provided custom fields, and pre-existing chapter files remain untouched unless explicitly superseded by newer content.

Practical Usage Examples

Create a new skill from a batch of analysis notes:

book-to-skill notes/*.txt my-research-skill

Fold new insights into an existing skill folder:

book-to-skill new-insights.md ~/.claude/skills/my-research-skill

Use the Python API directly for custom integrations:

from book_to_skill.cli import run

# Build a skill from text files and fold into existing structure

run(
    paths=["notes/section1.txt", "notes/section2.txt"],
    skill_name="my-research-skill",
    mode="fold",                     # mode 4 – fold into existing skill

    output_dir="~/.claude/skills"
)

Core Source Files and Architecture

Understanding the key components helps when extending or debugging the generating skills from existing analysis notes workflow:

• scripts/extract.py – Main orchestration entry point that coordinates file collection, parsing, sanitization, and skill assembly
• book_to_skill/parsers/text.py – Implements the text-specific parser responsible for identifying frameworks and chapter boundaries in plain-text notes
• book_to_skill/sanitize.py – Cleans raw input by removing dangerous Unicode controls and normalizing whitespace
• book_to_skill/utils.py – Provides file type detection and parser dispatch logic
• SKILL.md – The core descriptor residing inside each skill folder that stores metadata, frameworks, and routing information for agent invocation

Summary

• Fold-in mode automatically activates when the target directory contains SKILL.md and chapter-*.md files, enabling incremental updates to existing skills.
• The pipeline uses book_to_skill/sanitize.py to ensure all input text is safe UTF-8 before semantic processing occurs.
• Semantic extraction identifies mental models and frameworks, storing results in metadata.json with precise source attribution.
• New content merges into the existing skill structure without destroying custom user fields in SKILL.md or modifying untouched chapters.
• Temporary working files reside in /tmp/book_skill_work/ during processing and are cleaned automatically upon completion.

Frequently Asked Questions

How does book-to-skill determine whether to create a new skill or update an existing one?

The tool checks the target output path for the presence of a SKILL.md file and the characteristic chapter-*.md naming convention. When detected, scripts/extract.py switches to mode 4 (fold mode), triggering the merge logic that appends new analysis notes rather than initializing a fresh skill structure.

What file formats are supported for analysis notes?

According to the type detection logic in book_to_skill/utils.py, the system supports PDF, EPUB, DOCX, plain text (TXT), and Markdown files. Each format routes to a specialized parser within book_to_skill/parsers/, with text.py handling the standard text and Markdown inputs commonly used for analysis notes.

Where are temporary files stored during the skill generation process?

The pipeline uses /tmp/book_skill_work/ as a transient staging area to store full_text.txt, metadata.json, and intermediate chapter-<n>.md files before final assembly. This directory is automatically removed after the skill folder is successfully updated, ensuring no residual files remain on the system.

Will adding new notes overwrite my existing skill customizations?

No. The assembly logic specifically preserves user-provided custom fields in SKILL.md while only updating the metadata and framework sections derived from the new notes. Existing chapter files remain untouched unless the new input explicitly targets the same chapter identifiers for replacement.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}