How Book-to-Skill Handles CLI Parsing and Multi-Source Support

Book-to-Skill manually parses sys.argv in book_to_skill/cli.py to support flexible file specifications, expands globs and directories via resolve_input_files(), and merges multiple documents with visible source banners and consolidated metadata.

Book-to-Skill is an open-source document extraction tool that converts PDFs, EPUBs, and other formats into structured skill representations. Its command-line interface deliberately avoids heavy third-party parsing libraries in favor of a lightweight, manual argument processor capable of handling complex input specifications. Understanding how Book-to-Skill handles CLI parsing and multi-source support reveals an architecture optimized for reproducible batch processing and transparent data aggregation.

Manual Argument Parsing in book_to_skill/cli.py

The CLI entry point resides in the main() function, which delegates argument processing to parse_arguments(). Unlike typical Python CLIs that rely on argparse or click, this implementation manually walks the raw sys.argv list to maximize flexibility for mixed positional arguments and flags.

Handling Extraction Modes and Dependencies

The parser recognizes specific flags while collecting all non-flag tokens as input paths:

• --mode <technical|text> — Selects the extraction backend, with text as the default. Technical mode engages layout-aware PDF extraction via docling.
• --install-missing … — Controls how missing external dependencies are handled. The flag is accepted during parsing but its value is normalized later by normalize_install_mode.
• Unknown flags — Emit warnings rather than causing hard failures, maintaining backward compatibility.

Collecting Input Paths

All tokens not matching recognized flags are appended to an input_paths list. This design supports heterogeneous arguments where files, directories, and glob patterns can be interspersed freely without specific ordering constraints.

# Manual argument walking in parse_arguments()

# Source: book_to_skill/cli.py#L62-L96

Resolving and Normalizing Input Sources

After parsing, main() calls resolve_input_files() to transform user-provided strings into a deterministic, flat list of Path objects. This function handles the complexity of filesystem abstraction required for robust multi-source support.

Tilde Expansion and Glob Resolution

The implementation addresses shell-quoting issues and cross-platform path handling through explicit expansion:

• Tilde expansion — os.path.expanduser converts ~ into the user's home directory, ensuring patterns like ~/books/*.epub resolve correctly even when quoted in the shell.
• Glob expansion — Paths containing wildcard characters trigger glob.glob(..., recursive=True), enabling deep directory matching through patterns such as ./**/*.pdf.

Recursive Directory Traversal

When a path identifies as a directory, resolve_input_files() invokes os.walk to recurse through subdirectories. Only files with extensions appearing in SUPPORTED_EXTENSIONS (defined in book_to_skill/config.py) are collected, filtering out unsupported formats automatically.

Deterministic Sorting and Deduplication

To guarantee reproducible runs across different filesystems and operating systems:

1. Results are sorted case-insensitively before processing.
2. The final list undergoes deduplication while preserving the original order of explicit file arguments.
3. The output is a clean list of Path objects ready for sequential processing.

# Path resolution logic in resolve_input_files()

# Source: book_to_skill/cli.py#L98-L155

Multi-Source Processing and Metadata Merging

Once resolved, main() iterates over the file list and invokes extract_single_file() for each entry. When multiple sources are provided, Book-to-Skill implements sophisticated merging strategies to maintain data integrity.

Per-File Extraction

Each document is processed independently, with extraction results (raw text, metadata, structural analysis) stored temporarily in the extracted_sources collection. This isolation prevents format-specific parsing errors from cascading between files.

Text Combination with Source Demarcation

Raw texts are concatenated using visible banners formatted as === … SOURCE: <filename> === to clearly demarcate document boundaries in the combined output. This transparent delimiter system allows downstream consumers to identify content provenance.

Metadata Aggregation and Structural Detection

• Statistics merging — File sizes, page counts, and token estimates aggregate into a consolidated metadata dictionary representing the entire corpus.
• Table-of-Contents preservation — The system performs a boolean OR operation on per-source has_toc flags. If any individual document contains a detectable TOC, the combined output preserves this flag, ensuring structural information isn't lost when processing partial documents separately.

# Multi-source orchestration logic

# Source: book_to_skill/cli.py#L73-L115

Practical Usage Examples

Complex CLI Invocation

The following command demonstrates mixed input types, glob patterns, and explicit flags:

book-to-skill \
  ~/books/intro.pdf \
  ./chapter1.epub \
  ./reports/*.docx \
  --mode technical \
  --install-missing ask

Explanation:

• ~/books/intro.pdf — Tilde expansion resolves to the absolute home directory path.
• ./reports/*.docx — Glob expansion captures all Word documents in the reports folder.
• --mode technical — Engages layout-aware extraction for PDF documents.
• --install-missing ask — Enables interactive prompting for missing external tools.

Programmatic API Usage

Book-to-Skill functions can be imported directly for scripted automation:

import sys
from book_to_skill.cli import parse_arguments, resolve_input_files

# Simulate CLI invocation

sys.argv = [
    "book-to-skill",
    "mybook.epub",
    "chapter2.pdf",
    "--mode",
    "text",
]

paths, mode, install = parse_arguments(sys.argv)
files = resolve_input_files(paths)

print("Resolved files:", [f.name for f in files])
print("Extraction mode:", mode)
print("Install mode:", install)

Summary

• Zero-dependency parsing — parse_arguments() in book_to_skill/cli.py manually processes sys.argv to avoid external CLI framework dependencies.
• Flexible path resolution — resolve_input_files() supports tilde expansion (~), glob patterns (*, **), and recursive directory traversal while filtering against SUPPORTED_EXTENSIONS.
• Deterministic ordering — Input files are sorted case-insensitively and deduplicated to ensure reproducible processing across platforms.
• Intelligent merging — Multi-source inputs combine with visible source banners and consolidated metadata, with boolean OR logic preserving structural flags like has_toc across the corpus.

Frequently Asked Questions

Does Book-to-Skill use argparse or click for CLI parsing?

No. According to the virgiliojr94/book-to-skill source code, the tool implements a custom parser in parse_arguments() that manually iterates through sys.argv. This approach eliminates heavy dependencies and allows flexible mixing of flags and positional arguments without complex subparser configurations.

How does Book-to-Skill handle glob patterns and shell expansion?

The resolve_input_files() function explicitly calls os.path.expanduser for tilde handling and glob.glob(..., recursive=True) for wildcard expansion. This ensures patterns like ~/books/*.epub or ./data/**/*.pdf resolve correctly even when the shell passes quoted strings that prevent native glob expansion.

Can Book-to-Skill process entire directories recursively?

Yes. When a provided path is a directory, resolve_input_files() utilizes os.walk to recurse through subdirectories. It filters collected files against the SUPPORTED_EXTENSIONS tuple defined in book_to_skill/config.py, automatically excluding non-document files without raising errors.

How is metadata handled when merging multiple input documents?

The system aggregates per-source metadata into a unified dictionary while concatenating raw texts with visible source banners. For structural detection, it performs boolean OR operations on flags like has_toc across all inputs, ensuring that table-of-contents information detected in any single file persists in the combined output.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}