Book-to-Skill Directory Structure: A Complete Guide to Project Organization

The book-to-skill project follows a layered, modular directory structure that separates deterministic Python extraction logic from specification-driven skill generation and supporting tooling.

The virgiliojr94/book-to-skill repository is organized to cleanly distinguish between extracting content from books, generating AI-ready skills, and validating the results. Understanding this directory structure helps contributors and users navigate the codebase effectively.

Top-Level Directory Layout

The repository root contains these primary directories and files:

Entry	Purpose
SKILL.md	Core generator specification defining the 10-step agent workflow for building skills
scripts/	CLI entry point and the complete extraction engine
tools/	Helper utilities for cost measurement, validation, and security scanning
tests/	Pytest test suite covering extraction, sanitization, and discovery-tax calculations
docs/	Architecture, performance, usage, and FAQ documentation
Standard files	README.md, CONTRIBUTING.md, SECURITY.md, CHANGELOG.md

This separation ensures that extraction concerns remain isolated from generation logic, making the codebase maintainable and testable.

The scripts/ Directory: Extraction Engine

The scripts/ directory houses the deterministic Python extractor that converts books into structured text and metadata.

Entry Point

• extract.py – Thin CLI shim that forwards execution to the underlying package

Core Package Structure (scripts/extractor/)

Module	Responsibility
config.py	Extension mappings, paths, and dependency configuration map
dependencies.py	Optional-dependency probing with --check flag support
utils.py	CLI argument parsing, multi-source resolution, chapter detection, and runner orchestration
parsers/	Format-specific parsers for PDF, EPUB, DOCX, HTML, RTF, Calibre, and plain text

The parser modules in scripts/extractor/parsers/ include:

• pdf.py – PDF extraction (supports pdftotext, docling, and other backends)
• epub.py – EPUB e-book parsing
• docx.py – Microsoft Word document handling
• html.py – HTML document processing
• rtf.py – Rich Text Format support
• text.py – Plain text fallback

The tools/ Directory: Analysis and Validation

The tools/ directory contains standalone utilities that operate on extracted or generated artifacts:

• discovery_tax.py – Measures token cost versus full context-dump efficiency, helping optimize skill generation economics
• validate_skill.py – Validates a generated SKILL.md against host-specific rules (e.g., Copilot, Claude Code, etc.)

These tools are designed to run independently of the main extraction pipeline.

Data Flow: How the Directory Structure Supports the Pipeline

According to the architecture documentation in docs/architecture.md, the directory structure enables this workflow:

1. Extraction phase (scripts/extractor/): Source documents → full_text.txt + metadata.json
2. Generation phase: These outputs feed into the agent specified by SKILL.md
3. Deployment phase: Final skill files land in host-specific directories like ~/.copilot/skills/<slug>/

This architecture is visualized in the component map referenced in docs/architecture.md.

Complete Directory Tree

$ tree -L 3 .
├── SKILL.md                 # Generator specification (steps 0-10)

├── README.md
├── CONTRIBUTING.md
├── SECURITY.md
├── CHANGELOG.md
├── scripts/
│   ├── extract.py           # CLI entry shim

│   └── extractor/           # Core extraction package

│       ├── __init__.py
│       ├── config.py        # Path/extension/dependency config

│       ├── dependencies.py  # Optional dep probing (--check)

│       ├── utils.py         # CLI parsing, multi-source resolver

│       └── parsers/         # Format-specific extractors

│           ├── __init__.py
│           ├── pdf.py
│           ├── epub.py
│           ├── docx.py
│           ├── html.py
│           ├── rtf.py
│           ├── calibre.py
│           └── text.py
├── tools/
│   ├── discovery_tax.py     # Token cost measurement

│   └── validate_skill.py    # Host rule validation

├── tests/
│   ├── test_extraction.py
│   ├── test_sanitization.py
│   ├── test_discovery_tax.py
│   └── conftest.py
└── docs/
    ├── architecture.md      # Component diagrams and design

    ├── performance.md       # Benchmarks and optimization

    ├── usage.md             # End-user guides

    └── faq.md

Key Files and Their Roles

File Path	Function	Critical For
SKILL.md	Defines the 10-step agent workflow	Understanding how skills are generated
scripts/extract.py	Launches extraction via shim	Direct CLI invocation
scripts/extractor/utils.py	Argument parsing and orchestration	Custom integration
scripts/extractor/parsers/pdf.py	PDF content extraction	Working with scanned books
tools/validate_skill.py	Host compatibility checking	Pre-deployment verification
docs/architecture.md	System design documentation	Contributing or extending

Relationship Between Components

The book-to-skill directory structure enforces clean boundaries:

• Pure extraction lives in scripts/extractor/ with no knowledge of generation
• The root SKILL.md specifies generation without implementation details
• tools/ provide cross-cutting concerns (economics, security, validation)
• tests/ mirror the source structure for comprehensive coverage

Summary

• Top-level contains the generator spec (SKILL.md) and standard project metadata
• scripts/ hosts the deterministic Python extraction engine with modular parsers
• tools/ provides economic analysis (discovery_tax.py) and validation (validate_skill.py)
• docs/ includes architecture diagrams showing how extraction outputs feed into skill generation
• tests/ covers extraction, sanitization, and cost-calculation logic

Frequently Asked Questions

What is the purpose of SKILL.md in the root directory?

SKILL.md is the core generator specification that defines a 10-step agent workflow. It instructs the AI assistant how to transform extracted book content into a deployable skill, making it the central contract between extraction and generation phases.

Where does the actual PDF and EPUB parsing happen?

Format-specific parsing occurs in scripts/extractor/parsers/. Each supported format has its own module—pdf.py, epub.py, docx.py, etc.—allowing format detection to route content to the appropriate extractor with fallback options like pdftotext or docling for PDFs.

How can I validate a skill before deploying it?

Use tools/validate_skill.py, which checks a generated SKILL.md against host-specific rules for platforms like GitHub Copilot. This ensures compatibility before copying files to directories like ~/.copilot/skills/<slug>/.

What does discovery_tax.py measure?

tools/discovery_tax.py calculates the token cost efficiency of skill generation versus dumping full context, helping users optimize when skill-based retrieval provides economic advantages over raw context windows.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}