Where Is the Book-to-Skill Extractor Output Stored? A Complete Guide

The Book-to-Skill extractor writes its output to a temporary per-run work directory containing full_text.txt and metadata.json, defaulting to /tmp/book_skill_work-{PID}/ unless overridden by the BOOK_SKILL_WORKDIR environment variable.

The virgiliojr94/book-to-skill repository provides a document processing pipeline that extracts and consolidates text from multiple sources. Understanding where the Book-to-Skill extractor output is stored is critical for automation workflows, debugging extraction results, and securing sensitive document content. The storage location is determined dynamically at runtime through a combination of environment variables and secure temporary directory generation.

Default Output Directory Location

The extractor creates an isolated work directory for every invocation to prevent collisions between concurrent runs. In book_to_skill/config.py, the OUTPUT_DIR constant defines this location:

OUTPUT_DIR = Path(os.environ.get("BOOK_SKILL_WORKDIR") or default_output_dir())

When the BOOK_SKILL_WORKDIR environment variable is unset, the system falls back to default_output_dir(), which generates a process-specific temporary folder:

def default_output_dir() -> Path:
    return Path(tempfile.gettempdir()) / f"book_skill_work-{os.getpid()}"

This implementation guarantees that each process receives a unique directory based on its PID (Process ID), ensuring that parallel executions do not overwrite each other's data. On Linux systems, this typically resolves to paths like /tmp/book_skill_work-27491/, while Windows uses the %TEMP% equivalent.

Customizing the Output Path

You can override the default temporary location by setting the BOOK_SKILL_WORKDIR environment variable before running the extractor:

export BOOK_SKILL_WORKDIR=/var/lib/book_to_skill/output
book-to-skill ./my_documents

When this variable is present, OUTPUT_DIR resolves directly to the specified path without appending the PID suffix. This is essential for persistent storage in containerized environments or shared filesystems where temporary directories are ephemeral.

Output Files Generated

Inside the work directory defined by OUTPUT_DIR, the extractor produces exactly two files, both declared in book_to_skill/config.py:

• full_text.txt (OUTPUT_TEXT) – The concatenated, cleaned text extracted from all processed documents.
• metadata.json (OUTPUT_META) – A JSON object containing per-source statistics including file names, formats, page counts, and token estimates.

These paths are constructed as:

OUTPUT_TEXT = OUTPUT_DIR / "full_text.txt"
OUTPUT_META = OUTPUT_DIR / "metadata.json"

The actual write operation occurs in book_to_skill/utils.py within the main() function, where OUTPUT_TEXT.write_text() persists the consolidated content with UTF-8 encoding, followed by the metadata serialization.

Security and Directory Permissions

Before writing any data, the extractor invokes prepare_output_dir(OUTPUT_DIR) to enforce strict security constraints. This routine validates that the target directory is owned by the current user, is not a symlink, and sets filesystem permissions to 0700 (read/write/execute for owner only). This permission model protects extracted content from unauthorized access by other system users, which is vital when processing confidential or proprietary documents.

Accessing Output Programmatically

After extraction completes, you can locate and read the results using the configuration constants:

# Display the current output directory

python -c "import book_to_skill.config as c; print(c.OUTPUT_DIR)"

# View the extracted text

cat "$(python -c "import book_to_skill.config as c; print(c.OUTPUT_TEXT)")"

# Inspect metadata

cat "$(python -c "import book_to_skill.config as c; print(c.OUTPUT_META)")"

For Python integrations:

import json
from book_to_skill.config import OUTPUT_TEXT, OUTPUT_META

# After running book_to_skill.cli.main()

text_content = OUTPUT_TEXT.read_text(encoding="utf-8")
metadata = json.loads(OUTPUT_META.read_text(encoding="utf-8"))

print(f"Extracted {len(text_content)} characters from {len(metadata['source_file'])} source(s).")

Summary

• The Book-to-Skill extractor stores results in a per-run temporary directory defined by OUTPUT_DIR in book_to_skill/config.py.
• The default path uses the system temp folder with a PID suffix (/tmp/book_skill_work-{PID}/), ensuring isolation between concurrent processes.
• Set the BOOK_SKILL_WORKDIR environment variable to specify a custom, persistent output location.
• Each run generates full_text.txt and metadata.json, written by book_to_skill/utils.py after processing completes.
• The prepare_output_dir() function enforces 0700 permissions and symlink validation to secure sensitive extracted data.

Frequently Asked Questions

How do I change where the Book-to-Skill extractor saves its files?

Set the BOOK_SKILL_WORKDIR environment variable to your desired absolute path before executing the CLI. This overrides the default_output_dir() function and directs both full_text.txt and metadata.json to your specified location instead of the system temporary directory.

What files does the Book-to-Skill extractor create?

According to the source code in book_to_skill/config.py, the extractor generates exactly two files: full_text.txt (the consolidated, cleaned text content) and metadata.json (processing statistics and per-source information). These are referenced throughout the codebase as OUTPUT_TEXT and OUTPUT_META.

Is the output directory secure for sensitive documents?

Yes. The prepare_output_dir() function in book_to_skill/utils.py enforces ownership verification and sets directory permissions to 0700, meaning only the executing user can read, write, or execute within the folder. It also checks for symlink attacks to prevent path traversal vulnerabilities.

Can multiple instances run simultaneously without conflicts?

Yes. The default_output_dir() function incorporates os.getpid() into the directory name, creating unique paths like /tmp/book_skill_work-12345/ and /tmp/book_skill_work-12346/ for concurrent processes. This PID-based isolation prevents race conditions and file corruption during parallel extractions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}