# How Book-to-Skill Handles CLI Parsing and Multi-Source Support

> Learn how Book-to-Skill handles CLI parsing and multi-source support by manually parsing sys.argv, resolving input files, and merging documents with banners and metadata.

- Repository: [Virgilio Junior/book-to-skill](https://github.com/virgiliojr94/book-to-skill)
- Tags: internals
- Published: 2026-08-29

---

**Book-to-Skill manually parses `sys.argv` in [`book_to_skill/cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/cli.py) to support flexible file specifications, expands globs and directories via `resolve_input_files()`, and merges multiple documents with visible source banners and consolidated metadata.**

Book-to-Skill is an open-source document extraction tool that converts PDFs, EPUBs, and other formats into structured skill representations. Its command-line interface deliberately avoids heavy third-party parsing libraries in favor of a lightweight, manual argument processor capable of handling complex input specifications. Understanding how Book-to-Skill handles CLI parsing and multi-source support reveals an architecture optimized for reproducible batch processing and transparent data aggregation.

## Manual Argument Parsing in book_to_skill/cli.py

The CLI entry point resides in the `main()` function, which delegates argument processing to `parse_arguments()`. Unlike typical Python CLIs that rely on `argparse` or `click`, this implementation manually walks the raw `sys.argv` list to maximize flexibility for mixed positional arguments and flags.

### Handling Extraction Modes and Dependencies

The parser recognizes specific flags while collecting all non-flag tokens as **input paths**:

- **`--mode <technical|text>`** — Selects the extraction backend, with **text** as the default. Technical mode engages layout-aware PDF extraction via `docling`.
- **`--install-missing …`** — Controls how missing external dependencies are handled. The flag is accepted during parsing but its value is normalized later by `normalize_install_mode`.
- **Unknown flags** — Emit warnings rather than causing hard failures, maintaining backward compatibility.

### Collecting Input Paths

All tokens not matching recognized flags are appended to an `input_paths` list. This design supports heterogeneous arguments where files, directories, and glob patterns can be interspersed freely without specific ordering constraints.

```python

# Manual argument walking in parse_arguments()

# Source: book_to_skill/cli.py#L62-L96

```

## Resolving and Normalizing Input Sources

After parsing, `main()` calls `resolve_input_files()` to transform user-provided strings into a deterministic, flat list of `Path` objects. This function handles the complexity of filesystem abstraction required for robust multi-source support.

### Tilde Expansion and Glob Resolution

The implementation addresses shell-quoting issues and cross-platform path handling through explicit expansion:

- **Tilde expansion** — `os.path.expanduser` converts `~` into the user's home directory, ensuring patterns like `~/books/*.epub` resolve correctly even when quoted in the shell.
- **Glob expansion** — Paths containing wildcard characters trigger `glob.glob(..., recursive=True)`, enabling deep directory matching through patterns such as `./**/*.pdf`.

### Recursive Directory Traversal

When a path identifies as a directory, `resolve_input_files()` invokes `os.walk` to recurse through subdirectories. Only files with extensions appearing in `SUPPORTED_EXTENSIONS` (defined in [`book_to_skill/config.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/config.py)) are collected, filtering out unsupported formats automatically.

### Deterministic Sorting and Deduplication

To guarantee reproducible runs across different filesystems and operating systems:

1. Results are sorted **case-insensitively** before processing.
2. The final list undergoes deduplication while **preserving the original order** of explicit file arguments.
3. The output is a clean list of `Path` objects ready for sequential processing.

```python

# Path resolution logic in resolve_input_files()

# Source: book_to_skill/cli.py#L98-L155

```

## Multi-Source Processing and Metadata Merging

Once resolved, `main()` iterates over the file list and invokes `extract_single_file()` for each entry. When multiple sources are provided, Book-to-Skill implements sophisticated merging strategies to maintain data integrity.

### Per-File Extraction

Each document is processed independently, with extraction results (raw text, metadata, structural analysis) stored temporarily in the `extracted_sources` collection. This isolation prevents format-specific parsing errors from cascading between files.

### Text Combination with Source Demarcation

Raw texts are concatenated using visible banners formatted as `=== … SOURCE: <filename> ===` to clearly demarcate document boundaries in the combined output. This transparent delimiter system allows downstream consumers to identify content provenance.

### Metadata Aggregation and Structural Detection

- **Statistics merging** — File sizes, page counts, and token estimates aggregate into a consolidated metadata dictionary representing the entire corpus.
- **Table-of-Contents preservation** — The system performs a boolean **OR** operation on per-source `has_toc` flags. If any individual document contains a detectable TOC, the combined output preserves this flag, ensuring structural information isn't lost when processing partial documents separately.

```python

# Multi-source orchestration logic

# Source: book_to_skill/cli.py#L73-L115

```

## Practical Usage Examples

### Complex CLI Invocation

The following command demonstrates mixed input types, glob patterns, and explicit flags:

```bash
book-to-skill \
  ~/books/intro.pdf \
  ./chapter1.epub \
  ./reports/*.docx \
  --mode technical \
  --install-missing ask

```

**Explanation:**

- `~/books/intro.pdf` — Tilde expansion resolves to the absolute home directory path.
- `./reports/*.docx` — Glob expansion captures all Word documents in the reports folder.
- `--mode technical` — Engages layout-aware extraction for PDF documents.
- `--install-missing ask` — Enables interactive prompting for missing external tools.

### Programmatic API Usage

Book-to-Skill functions can be imported directly for scripted automation:

```python
import sys
from book_to_skill.cli import parse_arguments, resolve_input_files

# Simulate CLI invocation

sys.argv = [
    "book-to-skill",
    "mybook.epub",
    "chapter2.pdf",
    "--mode",
    "text",
]

paths, mode, install = parse_arguments(sys.argv)
files = resolve_input_files(paths)

print("Resolved files:", [f.name for f in files])
print("Extraction mode:", mode)
print("Install mode:", install)

```

## Summary

- **Zero-dependency parsing** — `parse_arguments()` in [`book_to_skill/cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/cli.py) manually processes `sys.argv` to avoid external CLI framework dependencies.
- **Flexible path resolution** — `resolve_input_files()` supports tilde expansion (`~`), glob patterns (`*`, `**`), and recursive directory traversal while filtering against `SUPPORTED_EXTENSIONS`.
- **Deterministic ordering** — Input files are sorted case-insensitively and deduplicated to ensure reproducible processing across platforms.
- **Intelligent merging** — Multi-source inputs combine with visible source banners and consolidated metadata, with boolean OR logic preserving structural flags like `has_toc` across the corpus.

## Frequently Asked Questions

### Does Book-to-Skill use argparse or click for CLI parsing?

No. According to the virgiliojr94/book-to-skill source code, the tool implements a custom parser in `parse_arguments()` that manually iterates through `sys.argv`. This approach eliminates heavy dependencies and allows flexible mixing of flags and positional arguments without complex subparser configurations.

### How does Book-to-Skill handle glob patterns and shell expansion?

The `resolve_input_files()` function explicitly calls `os.path.expanduser` for tilde handling and `glob.glob(..., recursive=True)` for wildcard expansion. This ensures patterns like `~/books/*.epub` or `./data/**/*.pdf` resolve correctly even when the shell passes quoted strings that prevent native glob expansion.

### Can Book-to-Skill process entire directories recursively?

Yes. When a provided path is a directory, `resolve_input_files()` utilizes `os.walk` to recurse through subdirectories. It filters collected files against the `SUPPORTED_EXTENSIONS` tuple defined in [`book_to_skill/config.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/config.py), automatically excluding non-document files without raising errors.

### How is metadata handled when merging multiple input documents?

The system aggregates per-source metadata into a unified dictionary while concatenating raw texts with visible source banners. For structural detection, it performs boolean OR operations on flags like `has_toc` across all inputs, ensuring that table-of-contents information detected in any single file persists in the combined output.