How book-to-skill Ensures Isolation Between Concurrent Extractions Using PIDs

book-to-skill prevents file collisions during simultaneous extractions by embedding the current process ID (PID) into a unique temporary work directory for each run.

The book-to-skill repository is a Python tool for extracting text and metadata from books. When multiple extraction jobs run concurrently, they risk overwriting each other's temporary files. The codebase solves this through PID-based directory isolation, a pattern implemented in book_to_skill/config.py and validated by dedicated tests.

How the PID-Based Directory Is Generated

The core isolation logic lives in book_to_skill/config.py. The default_output_dir() function constructs a path that guarantees uniqueness across processes:

import os
import tempfile
from pathlib import Path

def default_output_dir() -> Path:
    return Path(tempfile.gettempdir()) / f"book_skill_work-{os.getpid()}"

This returns a path like /tmp/book_skill_work-12345, where 12345 is the current process's PID. Since operating systems guarantee unique PIDs for active processes, concurrent extractions automatically receive distinct directories.

The OUTPUT_DIR constant applies this logic at module load time, with one safety refinement:

OUTPUT_DIR = Path(os.environ.get("BOOK_SKILL_WORKDIR") or default_output_dir())

The or operator—rather than dict.get()—ensures that an empty string environment variable falls back to the PID-based default. This prevents Path("") from resolving to the current working directory, which would pollute the project folder.

Files That Rely on This Isolation

Two key files consume OUTPUT_DIR to write extraction results:

• book_to_skill/cli.py — The CLI entry point that orchestrates the extraction pipeline
• scripts/extract.py — The script that writes full_text.txt and metadata.json into the work directory

Because both use the same OUTPUT_DIR from config.py, the entire extraction pipeline inherits PID-based isolation automatically.

Environment Variable Override Behavior

Users can bypass PID-based naming by setting BOOK_SKILL_WORKDIR:

export BOOK_SKILL_WORKDIR=/tmp/my_custom_dir
python -m book_to_skill extract mybook.pdf

The code handles three edge cases deliberately:

• Unset variable — Falls back to PID-based directory
• Valid path — Uses the specified directory directly
• Empty string — Treated as falsy, falls back to PID-based directory (not current directory)

Test Coverage for Isolation Guarantees

The test suite in tests/test_per_run_workdir.py verifies three critical properties:

1. Uniqueness per PID — The test test_default_workdir_is_unique_per_process monkey-patches os.getpid() to return different values and asserts the resulting directories differ:

# From tests/test_per_run_workdir.py lines 38-48

def test_default_workdir_is_unique_per_process(self, monkeypatch):
    monkeypatch.setattr(os, 'getpid', lambda: 1000)
    dir1 = default_output_dir()
    monkeypatch.setattr(os, 'getpid', lambda: 2000)
    dir2 = default_output_dir()
    assert dir1 != dir2
    assert "1000" in str(dir1)
    assert "2000" in str(dir2)

2. Sibling directory placement — The new PID-based path is a sibling of the legacy fixed path ($TMPDIR/book_skill_work), not a child. This prevents legacy cleanup routines from deleting active run directories.

3. Empty string handling — Tests confirm that BOOK_SKILL_WORKDIR="" does not collapse to the current working directory.

Practical Usage Examples

Access the configured directories in Python:

from book_to_skill.config import OUTPUT_DIR, OUTPUT_TEXT, OUTPUT_META

print("Work directory :", OUTPUT_DIR)      # e.g., /tmp/book_skill_work-12345

print("Full text file :", OUTPUT_TEXT)     # /tmp/book_skill_work-12345/full_text.txt

print("Metadata file :", OUTPUT_META)      # /tmp/book_skill_work-12345/metadata.json

Run two extractions concurrently without conflict:

import subprocess
import os

# Each subprocess gets its own PID and therefore its own work directory

subprocess.Popen(['python', '-m', 'book_to_skill', 'extract', 'book1.pdf'])
subprocess.Popen(['python', '-m', 'book_to_skill', 'extract', 'book2.pdf'])

Summary

• default_output_dir() in book_to_skill/config.py embeds os.getpid() into the directory name
• OUTPUT_DIR resolves at import time, ensuring consistent paths throughout the process lifetime
• BOOK_SKILL_WORKDIR provides optional override with safe empty-string handling
• Tests in tests/test_per_run_workdir.py verify PID uniqueness, directory placement, and edge cases
• Downstream files (cli.py, scripts/extract.py) inherit isolation without additional code

Frequently Asked Questions

What happens if two extractions run at exactly the same time?

Each Python process receives a unique PID from the operating system. Since default_output_dir() includes this PID in the path, simultaneous extractions create directories like /tmp/book_skill_work-12345 and /tmp/book_skill_work-12346—guaranteeing no file collisions.

Can I use a fixed directory for all extractions?

Yes. Set BOOK_SKILL_WORKDIR to your preferred path. The code uses this value directly, bypassing the PID-based naming. Be aware that concurrent runs with a fixed directory will overwrite each other's files unless you implement your own locking mechanism.

Why does the code use or instead of .get() with a default?

The expression os.environ.get("BOOK_SKILL_WORKDIR") or default_output_dir() ensures that an empty string environment variable evaluates as falsy and triggers the fallback. Using .get() with a default would return "", and Path("") resolves to the current working directory—an unsafe default that could pollute the project folder with temporary files.

How does this design affect legacy cleanup scripts?

The PID-based directory (book_skill_work-12345) is a sibling of the old fixed path (book_skill_work), not a subdirectory. Legacy scripts that delete book_skill_work recursively cannot accidentally destroy active extraction directories, since they reside at the same level with different names.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}