SKILL.md Generator Specification and Adaptive Token Budget Matrix Explained

The SKILL.md generator is a spec-driven agent that transforms book content into structured, on-demand skills using a three-dimensional adaptive token budget matrix to enforce size constraints across different book types, depth levels, and host environments.

The book-to-skill repository implements a novel approach to technical skill extraction: a deterministic Python extractor paired with a spec-driven generator that follows rigid rules defined in SKILL.md. This article breaks down the generator's architecture, its step-based pipeline, and the adaptive token budget matrix that keeps generated skills lean and performant.

Core Architecture of the SKILL.md Generator

The repository splits functionality into two distinct halves:

• Deterministic Extractor — scripts/extract.py and the book_to_skill/ package handle text extraction, cleaning, and metadata preservation
• Spec-Driven Generator — An agent that reads SKILL.md and executes a fixed 10-step pipeline (plus fold-in) to produce structured skill content

The generator is deliberately stateless. All information required to produce a skill lives inside the specification itself, making the system reproducible and host-agnostic. As documented in docs/architecture.md, this separation ensures the extractor remains deterministic while the generator adapts to different content types and user intents.

The 10-Step Generation Pipeline

The generator follows a numbered sequence defined in SKILL.md:

1. Step 0 — Initialize context and load book metadata
2. Steps 1-6 — Extract core concepts, structure, and relationships
3. Step 7 — Apply the adaptive token budget matrix (detailed below)
4. Steps 8-10 — Generate chapter files, glossary, patterns, and cheatsheet
5. Fold-in — Merge components into final deliverables

Each step produces intermediate artifacts that feed subsequent stages. The master SKILL.md serves as both specification and runtime guide, with the generator consulting it at every decision point.

Adaptive Token Budget Matrix Specification

The matrix is the generator's mechanism for keeping skills within operational limits. It operates across three dimensions with concrete token allocations:

Dimension	Value	Per-Chapter Budget
BOOK_TYPE	text (plain prose)	~1,000 tokens
	technical (code, tables, diagrams)	~1,800 tokens
DEPTH	reference (quick lookup)	Uses lower budget cell
	study (deep, worked-through)	Uses higher budget cell

Critical Budget Limits

According to SKILL.md (line 459), the core SKILL.md file must remain under 4,000 tokens — labeled the CRITICAL TOKEN BUDGET. Additional components carry their own constraints:

• Glossary, patterns, cheatsheet: ~4,500 tokens combined
• Per-chapter allocation: Determined by matrix lookup
• Total formula (line 207): chapters × per_chapter_budget + 4,000 + 4,500

Adaptation Logic

The matrix responds to two runtime signals:

1. Explicit user intent — When a user requests "study this chapter," the generator selects the study depth column
2. Content inference — For mixed prose/technical material, the generator defaults to the lower budget and infers depth from content richness (line 354)

This adaptation prevents over-allocation while preserving essential technical detail.

Budget Selection Implementation

The generator implements matrix lookup through conditional evaluation. While the full source uses the spec parser, the selection logic follows this pattern:

# Simplified matrix lookup from SKILL.md specification

def select_chapter_budget(book_type: str, depth: str) -> int:
    """
    Returns per-chapter token budget based on adaptive matrix.
    Spec reference: SKILL.md lines 345-356
    """
    base_budgets = {
        "text": 1000,        # Plain prose baseline

        "technical": 1800    # Code-heavy baseline

    }
    
    base = base_budgets.get(book_type, 1000)
    
    if depth == "study":
        return base          # Full budget for deep engagement

    else:  # reference

        return int(base * 0.7)  # Compressed for quick lookup

For mixed-content books, the generator applies conservative defaults:

# Mixed content fallback (SKILL.md line 354)

if content_has_mixed_types(book_content):
    effective_type = "text"  # Lower budget wins

    inferred_depth = infer_depth_from_richness(book_content)
    budget = select_chapter_budget(effective_type, inferred_depth)

Validation Against Host Constraints

The tools/validate_skill.py script enforces budget compliance across supported hosts. As referenced in AGENTS.md (line 83) and CONTRIBUTING.md (line 33), validation accepts a --lens parameter:

# Validate against Claude's context window

python3 tools/validate_skill.py generated/SKILL.md --lens claude

# Validate against Copilot constraints

python3 tools/validate_skill.py generated/SKILL.md --lens copilot

# Validate against AMP (Amazon) limits

python3 tools/validate_skill.py generated/SKILL.md --lens amp

The validator checks:

• Core SKILL.md ≤ 4,000 tokens
• Chapter files ≤ matrix-determined budget
• Total skill footprint within host-specific limits

File Structure and Responsibilities

File	Role	Key Contribution
SKILL.md	Master specification	Defines steps, matrix values, budget formulas (lines 207, 345-356, 459)
docs/architecture.md	System documentation	Extractor ↔ generator pipeline diagram
tools/validate_skill.py	Quality gate	Host-specific token limit enforcement
book_to_skill/cli.py	Orchestration entry point	Coordinates extraction → generation workflow
scripts/extract.py	Content preparation	Clean text + metadata for generator consumption

Summary

• The SKILL.md generator is a spec-driven, stateless agent executing a 10-step pipeline to convert books into structured skills
• The adaptive token budget matrix combines BOOK_TYPE (text/technical) and DEPTH (reference/study) to determine per-chapter allocations of 1,000–1,800 tokens
• Critical limits enforce 4,000 tokens for core SKILL.md and ~4,500 tokens for auxiliary assets
• Mixed-content adaptation defaults to conservative budgets with depth inferred from material richness
• Validation tooling ensures compliance across Claude, Copilot, and AMP host environments

Frequently Asked Questions

How does the generator choose between reference and study depth modes?

The generator selects depth based on explicit user intent when available — for example, a command like "study chapter 5" triggers study mode with full budget allocation. When intent is unspecified, the generator analyzes content richness (density of concepts, code examples, cross-references) to infer appropriate depth, defaulting to reference mode for safety.

What happens when a book contains both prose and technical content?

The specification mandates conservative budgeting for mixed-content books. According to SKILL.md (line 354), the generator uses the lower budget cell (treating the book as text type) and infers depth from the technical richness present. This prevents overspending tokens on chapters that may appear prose-heavy but contain scattered technical sections.

Why is the core SKILL.md limited to 4,000 tokens specifically?

The 4,000-token CRITICAL TOKEN BUDGET ensures the master specification loads instantly into any supported host's context window without truncation. This "front-loaded" design keeps essential knowledge immediately accessible while deferring chapter-specific detail to on-demand loading — crucial for maintaining responsiveness in conversational AI interfaces.

Where is the token budget matrix actually defined in the codebase?

The matrix parameters reside in SKILL.md itself, not in executable Python code. Lines 345-356 define the two-dimensional matrix structure, line 207 provides the total budget formula, and line 459 specifies the critical 4,000-token limit. The generator parses these values at runtime, making SKILL.md both specification and configuration source.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}