# SKILL.md Generator Specification and Adaptive Token Budget Matrix Explained

> Understand the SKILL.md generator spec and adaptive token budget matrix. Convert book content to structured skills with size constraints for diverse environments. Learn more!

- Repository: [Virgilio Junior/book-to-skill](https://github.com/virgiliojr94/book-to-skill)
- Tags: deep-dive
- Published: 2026-08-25

---

**The SKILL.md generator is a spec-driven agent that transforms book content into structured, on-demand skills using a three-dimensional adaptive token budget matrix to enforce size constraints across different book types, depth levels, and host environments.**

The `book-to-skill` repository implements a novel approach to technical skill extraction: a deterministic Python extractor paired with a **spec-driven generator** that follows rigid rules defined in [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md). This article breaks down the generator's architecture, its step-based pipeline, and the adaptive token budget matrix that keeps generated skills lean and performant.

## Core Architecture of the SKILL.md Generator

The repository splits functionality into two distinct halves:

- **Deterministic Extractor** — [`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py) and the `book_to_skill/` package handle text extraction, cleaning, and metadata preservation
- **Spec-Driven Generator** — An agent that reads [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) and executes a fixed 10-step pipeline (plus fold-in) to produce structured skill content

The generator is **deliberately stateless**. All information required to produce a skill lives inside the specification itself, making the system reproducible and host-agnostic. As documented in [`docs/architecture.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/docs/architecture.md), this separation ensures the extractor remains deterministic while the generator adapts to different content types and user intents.

## The 10-Step Generation Pipeline

The generator follows a numbered sequence defined in [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md):

1. **Step 0** — Initialize context and load book metadata
2. **Steps 1-6** — Extract core concepts, structure, and relationships
3. **Step 7** — Apply the **adaptive token budget matrix** (detailed below)
4. **Steps 8-10** — Generate chapter files, glossary, patterns, and cheatsheet
5. **Fold-in** — Merge components into final deliverables

Each step produces intermediate artifacts that feed subsequent stages. The master [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) serves as both specification and runtime guide, with the generator consulting it at every decision point.

## Adaptive Token Budget Matrix Specification

The matrix is the generator's mechanism for keeping skills within operational limits. It operates across **three dimensions** with concrete token allocations:

| Dimension | Value | Per-Chapter Budget |
|-----------|-------|-------------------|
| **BOOK_TYPE** | `text` (plain prose) | ~1,000 tokens |
| | `technical` (code, tables, diagrams) | ~1,800 tokens |
| **DEPTH** | `reference` (quick lookup) | Uses lower budget cell |
| | `study` (deep, worked-through) | Uses higher budget cell |

### Critical Budget Limits

According to [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) (line 459), the core [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) file must remain under **4,000 tokens** — labeled the **CRITICAL TOKEN BUDGET**. Additional components carry their own constraints:

- **Glossary, patterns, cheatsheet**: ~4,500 tokens combined
- **Per-chapter allocation**: Determined by matrix lookup
- **Total formula** (line 207): `chapters × per_chapter_budget + 4,000 + 4,500`

### Adaptation Logic

The matrix responds to two runtime signals:

1. **Explicit user intent** — When a user requests "study this chapter," the generator selects the `study` depth column
2. **Content inference** — For mixed prose/technical material, the generator defaults to the **lower budget** and infers depth from content richness (line 354)

This adaptation prevents over-allocation while preserving essential technical detail.

## Budget Selection Implementation

The generator implements matrix lookup through conditional evaluation. While the full source uses the spec parser, the selection logic follows this pattern:

```python

# Simplified matrix lookup from SKILL.md specification

def select_chapter_budget(book_type: str, depth: str) -> int:
    """
    Returns per-chapter token budget based on adaptive matrix.
    Spec reference: SKILL.md lines 345-356
    """
    base_budgets = {
        "text": 1000,        # Plain prose baseline

        "technical": 1800    # Code-heavy baseline

    }
    
    base = base_budgets.get(book_type, 1000)
    
    if depth == "study":
        return base          # Full budget for deep engagement

    else:  # reference

        return int(base * 0.7)  # Compressed for quick lookup

```

For mixed-content books, the generator applies conservative defaults:

```python

# Mixed content fallback (SKILL.md line 354)

if content_has_mixed_types(book_content):
    effective_type = "text"  # Lower budget wins

    inferred_depth = infer_depth_from_richness(book_content)
    budget = select_chapter_budget(effective_type, inferred_depth)

```

## Validation Against Host Constraints

The [`tools/validate_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/validate_skill.py) script enforces budget compliance across supported hosts. As referenced in [`AGENTS.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/AGENTS.md) (line 83) and [`CONTRIBUTING.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/CONTRIBUTING.md) (line 33), validation accepts a `--lens` parameter:

```bash

# Validate against Claude's context window

python3 tools/validate_skill.py generated/SKILL.md --lens claude

# Validate against Copilot constraints

python3 tools/validate_skill.py generated/SKILL.md --lens copilot

# Validate against AMP (Amazon) limits

python3 tools/validate_skill.py generated/SKILL.md --lens amp

```

The validator checks:
- Core [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) ≤ 4,000 tokens
- Chapter files ≤ matrix-determined budget
- Total skill footprint within host-specific limits

## File Structure and Responsibilities

| File | Role | Key Contribution |
|------|------|----------------|
| [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) | Master specification | Defines steps, matrix values, budget formulas (lines 207, 345-356, 459) |
| [`docs/architecture.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/docs/architecture.md) | System documentation | Extractor ↔ generator pipeline diagram |
| [`tools/validate_skill.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tools/validate_skill.py) | Quality gate | Host-specific token limit enforcement |
| [`book_to_skill/cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/cli.py) | Orchestration entry point | Coordinates extraction → generation workflow |
| [`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py) | Content preparation | Clean text + metadata for generator consumption |

## Summary

- The **SKILL.md generator** is a spec-driven, stateless agent executing a 10-step pipeline to convert books into structured skills
- The **adaptive token budget matrix** combines `BOOK_TYPE` (text/technical) and `DEPTH` (reference/study) to determine per-chapter allocations of 1,000–1,800 tokens
- **Critical limits** enforce 4,000 tokens for core [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) and ~4,500 tokens for auxiliary assets
- **Mixed-content adaptation** defaults to conservative budgets with depth inferred from material richness
- **Validation tooling** ensures compliance across Claude, Copilot, and AMP host environments

## Frequently Asked Questions

### How does the generator choose between reference and study depth modes?

The generator selects depth based on explicit user intent when available — for example, a command like "study chapter 5" triggers `study` mode with full budget allocation. When intent is unspecified, the generator analyzes content richness (density of concepts, code examples, cross-references) to infer appropriate depth, defaulting to `reference` mode for safety.

### What happens when a book contains both prose and technical content?

The specification mandates conservative budgeting for mixed-content books. According to [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) (line 354), the generator uses the **lower budget cell** (treating the book as `text` type) and infers depth from the technical richness present. This prevents overspending tokens on chapters that may appear prose-heavy but contain scattered technical sections.

### Why is the core SKILL.md limited to 4,000 tokens specifically?

The 4,000-token **CRITICAL TOKEN BUDGET** ensures the master specification loads instantly into any supported host's context window without truncation. This "front-loaded" design keeps essential knowledge immediately accessible while deferring chapter-specific detail to on-demand loading — crucial for maintaining responsiveness in conversational AI interfaces.

### Where is the token budget matrix actually defined in the codebase?

The matrix parameters reside in [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) itself, not in executable Python code. Lines 345-356 define the two-dimensional matrix structure, line 207 provides the total budget formula, and line 459 specifies the critical 4,000-token limit. The generator parses these values at runtime, making [`SKILL.md`](https://github.com/virgiliojr94/book-to-skill/blob/main/SKILL.md) both specification and configuration source.