# How to Extend book-to-skill by Adding a New Format Parser: A Step-by-Step Guide

> Extend book-to-skill by adding a new format parser. Follow this guide to implement extraction functions, register extensions, and test your integration in the virgiliojr94/book-to-skill repository.

- Repository: [Virgilio Junior/book-to-skill](https://github.com/virgiliojr94/book-to-skill)
- Tags: how-to-guide
- Published: 2026-08-25

---

**To extend book-to-skill with a new format parser, create a Python module in `book_to_skill/parsers/` that implements an extraction function, register the file extension in [`utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/utils.py), expose the module in [`parsers/__init__.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/parsers/__init__.py), and add tests to verify integration.**

`book-to-skill` is an open-source document processing tool that extracts raw text from multiple source formats through a clean, modular parser architecture. The parser layer lives in `book_to_skill/parsers/` and follows a simple contract: each module exposes a function that receives a file path and returns extracted text or `None` on failure. Understanding how to extend this system lets developers add support for any custom document format without modifying core business logic.

## How the Parser Architecture Works

The extraction pipeline centers on `extract_single_file` in [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py). This dispatcher inspects file extensions, looks up the corresponding parser in a registry dictionary, and delegates extraction to the appropriate module. This design keeps format-specific code isolated and makes adding new parsers a matter of implementing three touch-points: the parser module, the registry entry, and package exports.

Key reference implementations demonstrate the pattern:

- **[`book_to_skill/parsers/text.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/parsers/text.py)** — Minimal example with BOM handling and encoding detection
- **[`book_to_skill/parsers/pdf.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/parsers/pdf.py)** — Multiple fallback extraction strategies
- **[`book_to_skill/parsers/html.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/parsers/html.py)** — HTML-specific cleaning and normalization

## Step 1: Create the Parser Module

Create a new `.py` file in `book_to_skill/parsers/` following the established function signature. The function must accept a path string and return `str | None`.

```python

# book_to_skill/parsers/myformat.py

from __future__ import annotations
import sys
from pathlib import Path

def extract_myformat(path: str) -> str | None:
    """Extract raw text from a *.myfmt file.

    Returns the extracted string or ``None`` on failure.
    """
    try:
        data = Path(path).read_bytes()
    except Exception as e:
        print(f"  [warn] extract_myformat failed: {type(e).__name__}: {e}",
              file=sys.stderr)
        return None

    # Example: binary format starts with a custom BOM.

    if data.startswith(b"\x01\x02\x03\x04"):
        try:
            return data.decode("utf-8")
        except UnicodeDecodeError:
            pass

    # Fallback – try common encodings.

    for enc in ("utf-8", "cp1252", "latin-1"):
        try:
            return data.decode(enc)
        except UnicodeDecodeError:
            continue
    return None

```

**Design recommendations:**

- Handle binary and text formats gracefully
- Return `None` rather than raising exceptions on unrecoverable errors
- Print warnings to `stderr` for debugging without breaking downstream processing
- Provide encoding fallbacks for text-based formats

## Step 2: Register in the Parser Registry

Open [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py) and locate the `PARSER_BY_EXT` dictionary. Import your new module and add an entry mapping the file extension to your extraction function.

```python

# book_to_skill/utils.py

from .parsers import myformat

PARSER_BY_EXT = {
    ".txt": parsers.text.read_text_file,
    ".pdf": parsers.pdf.extract_pdf,
    ".html": parsers.html.extract_html_file,
    ".epub": parsers.epub.extract_epub,
    ".docx": parsers.docx.extract_docx,
    ".rtf": parsers.rtf.extract_rtf,
    ".myfmt": myformat.extract_myformat,   # ← new entry

}

```

The registry is the only place where your new parser connects to the core extraction pipeline. This single-point registration keeps dependencies explicit and reviewable.

## Step 3: Expose the Module in Package Exports

Update [`book_to_skill/parsers/__init__.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/parsers/__init__.py) to include your module in the package's public interface:

```python

# book_to_skill/parsers/__init__.py

from . import text, pdf, html, epub, docx, rtf, calibre, myformat

```

This import ensures that [`utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/utils.py) can resolve the module when it performs its lookup during extraction.

## Step 4: Update CLI Help Text (Optional)

In [`book_to_skill/cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/cli.py), locate where supported formats are listed for user help. Add your new extension to maintain accurate documentation:

```python

# Example location in cli.py

SUPPORTED_FORMATS = [".txt", ".pdf", ".html", ".epub", ".docx", ".rtf", ".myfmt"]

```

This step improves discoverability and prevents user confusion about which formats the tool accepts.

## Step 5: Write Integration Tests

Add a test file under `tests/` that exercises your parser through the full extraction pipeline:

```python

# tests/test_myformat_parser.py

def test_myformat_extraction(tmp_path):
    sample = tmp_path / "sample.myfmt"
    sample.write_bytes(b"\x01\x02\x03\x04Hello world!")
    from book_to_skill.utils import extract_single_file
    result = extract_single_file(
        sample,
        extraction_mode="myfmt",
        install_mode="none"
    )
    assert "Hello world!" in result["text"]

```

Run the test suite to verify your implementation:

```bash
pytest -q tests/test_myformat_parser.py

```

## Step 6: Validate with Repository Checks

Before submitting changes, ensure code quality and compatibility:

```bash
ruff check .          # Static analysis

pytest -q             # Full test suite

```

These checks confirm that your new parser integrates cleanly without regressions.

## Key Files for Parser Development

| File | Purpose |
|------|---------|
| `book_to_skill/parsers/` | Directory containing all format-specific extractors |
| [`book_to_skill/parsers/text.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/parsers/text.py) | Minimal reference implementation |
| [`book_to_skill/parsers/pdf.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/parsers/pdf.py) | Complex parser with multiple strategies |
| [`book_to_skill/utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/utils.py) | Core dispatcher and `PARSER_BY_EXT` registry |
| [`book_to_skill/cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/cli.py) | Command-line interface and help text |
| `tests/` | Test fixtures and validation suite |

## Summary

- **Create** a parser module in `book_to_skill/parsers/` with an `extract_<fmt>(path: str) -> str | None` function
- **Register** the extension-to-function mapping in [`utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/utils.py)'s `PARSER_BY_EXT` dictionary
- **Export** the module in [`parsers/__init__.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/parsers/__init__.py) for package visibility
- **Document** the new format in [`cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/cli.py) help text
- **Test** with pytest using the `extract_single_file` utility to verify end-to-end integration
- **Validate** with `ruff` and the full test suite before contribution

The parser layer's deterministic, isolated design guarantees that adding a new format parser touches only the registry and optional CLI documentation—no core logic changes required.

## Frequently Asked Questions

### What function signature must a custom parser implement?

A custom parser must implement `def extract_<fmt>(path: str) -> str | None:` where `path` is the absolute file path and the return value is the extracted text string or `None` if extraction fails. This signature matches all built-in parsers in `book_to_skill/parsers/` and ensures compatibility with the `extract_single_file` dispatcher.

### Can a parser return metadata alongside text?

The base contract returns only text, but you can extend the return structure if downstream code is updated accordingly. The current `extract_single_file` implementation in [`utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/utils.py) expects a string return and packages it into a result dictionary. For metadata-rich formats, consider returning a structured string or modifying the dispatcher to handle dictionary returns.

### How does book-to-skill handle parser failures gracefully?

Each parser catches exceptions internally, prints warnings to `stderr`, and returns `None` on failure. The `extract_single_file` function in [`utils.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/utils.py) handles `None` returns by propagating empty results or fallback behavior, ensuring that one parser failure does not crash the entire extraction pipeline.

### Is it possible to override an existing parser for a format?

Yes. Simply register your replacement function in `PARSER_BY_EXT` using the same extension key. The dictionary lookup uses the last-assigned value, so your custom parser will supersede the built-in implementation. This technique enables patching or enhancing extraction for problematic formats without forking the repository.