# How to Update and Fold‑In New Chapters When Merging Them into an Existing Skill

> Learn the update fold in workflow for merging new chapters into an existing skill. This guide covers preparing source files, running the extractor, and validating the updated SKILL.md.

- Repository: [Virgilio Junior/book-to-skill](https://github.com/virgiliojr94/book-to-skill)
- Tags: how-to-guide
- Published: 2026-08-25

---

**Merging new chapters into an existing book‑to‑skill skill follows a three‑stage workflow: prepare source files, run the extractor with the `--fold-in` flag, then validate and commit the updated [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md).**

The `virgiliojr94/book-to-skill` repository provides a deterministic pipeline for converting book content into structured skills. When you need to incorporate additional chapters into an already‑established skill, the **fold‑in workflow** ensures clean integration without overwriting existing content. This guide walks through the exact process as implemented in the source code.

## Understanding the Fold‑In Architecture

The fold‑in mechanism separates content preparation from structural merging. Unlike a full regeneration that rebuilds the entire skill from scratch, fold‑in preserves your existing [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) structure while surgically inserting new chapter blocks.

Two core components handle this:

- **[`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py)** – The CLI entry point that orchestrates extraction and accepts the `--fold-in` flag.

- **`book_to_skill/parsers/*.py`** – Format‑specific parsers that generate markdown fragments and determine insertion points within the existing hierarchy.

## Stage 1: Prepare New Chapter Sources

Before running any commands, place your new chapter files in a dedicated directory. The extractor accepts multiple formats including PDF, EPUB, DOCX, HTML, and RTF.

```bash

# Create a staging directory for new content

mkdir -p new_chapter

# Add your source files (example: PDF and EPUB)

curl -L -o new_chapter/chapter7.pdf https://example.com/chapter7.pdf
cp /path/to/chapter8.epub new_chapter/

```

Optionally, include a [`metadata.yaml`](https://github.com/virgiliojr94/book-to-skill/blob/main/metadata.yaml) file to specify chapter order, custom tags, or explicit placement instructions. The configuration system in [`book_to_skill/config.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/config.py) reads these values to influence parser behavior.

```yaml

# new_chapter/metadata.yaml

title: "Chapter 7: Advanced Topics"
order: 7
tags: ["advanced", "optimization"]

```

## Stage 2: Execute the Fold‑In Command

The critical operation uses the **`--fold-in`** flag, pointing the extractor at both your existing skill directory and the new source files.

```bash
python -m book_to_skill extract \
    --fold-in skills/my_skill/ \
    new_chapter/*

```

Here's what happens under the hood:

1. **Format detection** – Each parser in `book_to_skill/parsers/` identifies file types and extracts raw text.

2. **Sanitization** – [`book_to_skill/sanitize.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/sanitize.py) strips unsafe Unicode (BIDI controls, zero‑width spaces) and normalizes line endings to prevent malformed markdown.

3. **Structural insertion** – Parsers analyze heading hierarchies in the existing [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) and insert new sections at the appropriate depth, preserving table‑of‑contents integrity as defined in [`docs/architecture.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/docs/architecture.md).

4. **Idempotent output** – The merge is deterministic; identical inputs produce identical output, which is essential for CI reproducibility.

## Stage 3: Validate and Commit Changes

After extraction, verify structural integrity before committing.

```bash

# Validate the updated skill against the architectural contract

python tools/validate_skill.py skills/my_skill/SKILL.md

```

The validator enforces:

- Proper markdown fence structure
- Heading hierarchy compliance
- Schema adherence as documented in **[`docs/architecture.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/docs/architecture.md)**

Once validation passes, stage and commit with a descriptive message:

```bash
git add skills/my_skill/SKILL.md
git commit -m "Fold-in Chapter 7: Advanced Topics"

```

The repository's CI pipeline ([`.github/workflows/ci.yml`](https://github.com/virgiliojr94/book-to-skill/blob/main/.github/workflows/ci.yml)) automatically runs `pytest -q` and `ruff check .` to guard against regressions.

## Complete Workflow Example

```bash

# 1. Prepare sources

mkdir -p new_chapter
curl -L -o new_chapter/chapter7.pdf https://example.com/chapter7.pdf
curl -L -o new_chapter/chapter8.epub https://example.com/chapter8.epub

# 2. Fold into existing skill

python -m book_to_skill extract \
    --fold-in skills/my_skill/ \
    new_chapter/*

# 3. Validate

python tools/validate_skill.py skills/my_skill/SKILL.md

# 4. Run local quality checks

pytest -q
ruff check .

# 5. Commit

git add skills/my_skill/SKILL.md
git commit -m "Fold-in Chapters 7-8: Advanced Topics and Case Studies"

```

## Key Source Files and Their Roles

| File | Purpose | Direct Link |
|------|---------|-------------|
| [`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py) | CLI entry point; handles `--fold-in` flag | [extract.py](https://github.com/virgiliojr94/book-to-skill/blob/master/scripts/extract.py) |
| `book_to_skill/parsers/*.py` | Format‑specific extraction and insertion logic | [parsers/](https://github.com/virgiliojr94/book-to-skill/tree/master/book_to_skill/parsers) |
| [`book_to_skill/sanitize.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/sanitize.py) | Text normalization and unsafe character removal | [sanitize.py](https://github.com/virgiliojr94/book-to-skill/blob/master/book_to_skill/sanitize.py) |
| [`book_to_skill/config.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/config.py) | Metadata and configuration defaults | [config.py](https://github.com/virgiliojr94/book-to-skill/blob/master/book_to_skill/config.py) |
| [`tools/validate_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/validate_skill.py) | Structural validation against architecture spec | [validate_skill.py](https://github.com/virgiliojr94/book-to-skill/blob/master/tools/validate_skill.py) |
| [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) | Canonical skill definition (updated by fold‑in) | [SKILL.md](https://github.com/virgiliojr94/book-to-skill/blob/master/SKILL.md) |
| [`docs/architecture.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/docs/architecture.md) | Skill structure contract (headings, TOC, metadata) | [architecture.md](https://github.com/virgiliojr94/book-to-skill/blob/master/docs/architecture.md) |
| [`.github/workflows/ci.yml`](https://github.com/virgiliojr94/book-to-skill/blob/main/.github/workflows/ci.yml) | Automated testing and linting post‑merge | [ci.yml](https://github.com/virgiliojr94/book-to-skill/blob/master/.github/workflows/ci.yml) |

## Summary

- **Prepare** new chapters by placing source files (PDF, EPUB, DOCX, etc.) in a staging directory with optional [`metadata.yaml`](https://github.com/virgiliojr94/book-to-skill/blob/main/metadata.yaml).
- **Fold‑in** using `python -m book_to_skill extract --fold-in <skill_dir> <sources>` — this preserves existing [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) structure while inserting new content.
- **Validate** with [`tools/validate_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/validate_skill.py) to ensure compliance with the architectural contract.
- **Commit** only after local tests pass (`pytest`, `ruff`) to maintain CI pipeline health.

## Frequently Asked Questions

### What file formats does the fold‑in workflow support?

The extractor supports PDF, EPUB, DOCX, HTML, and RTF through dedicated parsers in `book_to_skill/parsers/`. Each parser normalizes extracted content to markdown fragments before insertion.

### Why use `--fold-in` instead of regenerating the entire skill?

Fold‑in preserves existing structural edits, metadata annotations, and manual formatting in your [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md). Full regeneration rebuilds from source files only, discarding any hand‑tuned changes you made to the skill output.

### How does the validator ensure my skill remains compliant?

[`tools/validate_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/validate_skill.py) enforces the schema defined in [`docs/architecture.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/docs/architecture.md), checking heading depth, TOC consistency, markdown fence integrity, and metadata requirements. It catches structural errors before they break downstream consumers.

### What happens if the same fold‑in command runs twice?

The process is **idempotent** — identical inputs produce identical [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) output. This determinism enables reliable CI checks and prevents duplicate content insertion during automated workflows.