# How book-to-skill Ensures Isolation Between Concurrent Extractions Using PIDs

> Discover how book-to-skill uses PIDs to isolate concurrent extractions, preventing file collisions by creating unique temporary directories for each process.

- Repository: [Virgilio Junior/book-to-skill](https://github.com/virgiliojr94/book-to-skill)
- Tags: internals
- Published: 2026-08-25

---

**book-to-skill** prevents file collisions during simultaneous extractions by embedding the current process ID (PID) into a unique temporary work directory for each run.

The **book-to-skill** repository is a Python tool for extracting text and metadata from books. When multiple extraction jobs run concurrently, they risk overwriting each other's temporary files. The codebase solves this through **PID-based directory isolation**, a pattern implemented in [`book_to_skill/config.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/config.py) and validated by dedicated tests.

## How the PID-Based Directory Is Generated

The core isolation logic lives in [`book_to_skill/config.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/config.py). The `default_output_dir()` function constructs a path that guarantees uniqueness across processes:

```python
import os
import tempfile
from pathlib import Path

def default_output_dir() -> Path:
    return Path(tempfile.gettempdir()) / f"book_skill_work-{os.getpid()}"

```

This returns a path like `/tmp/book_skill_work-12345`, where `12345` is the current process's PID. Since operating systems guarantee unique PIDs for active processes, concurrent extractions automatically receive distinct directories.

The `OUTPUT_DIR` constant applies this logic at module load time, with one safety refinement:

```python
OUTPUT_DIR = Path(os.environ.get("BOOK_SKILL_WORKDIR") or default_output_dir())

```

The `or` operator—rather than `dict.get()`—ensures that an empty string environment variable falls back to the PID-based default. This prevents `Path("")` from resolving to the current working directory, which would pollute the project folder.

## Files That Rely on This Isolation

Two key files consume `OUTPUT_DIR` to write extraction results:

- **[`book_to_skill/cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/cli.py)** — The CLI entry point that orchestrates the extraction pipeline
- **[`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py)** — The script that writes [`full_text.txt`](https://github.com/virgiliojr94/book-to-skill/blob/main/full_text.txt) and [`metadata.json`](https://github.com/virgiliojr94/book-to-skill/blob/main/metadata.json) into the work directory

Because both use the same `OUTPUT_DIR` from [`config.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/config.py), the entire extraction pipeline inherits PID-based isolation automatically.

## Environment Variable Override Behavior

Users can bypass PID-based naming by setting **`BOOK_SKILL_WORKDIR`**:

```bash
export BOOK_SKILL_WORKDIR=/tmp/my_custom_dir
python -m book_to_skill extract mybook.pdf

```

The code handles three edge cases deliberately:

- **Unset variable** — Falls back to PID-based directory
- **Valid path** — Uses the specified directory directly
- **Empty string** — Treated as falsy, falls back to PID-based directory (not current directory)

## Test Coverage for Isolation Guarantees

The test suite in [`tests/test_per_run_workdir.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tests/test_per_run_workdir.py) verifies three critical properties:

1. **Uniqueness per PID** — The test `test_default_workdir_is_unique_per_process` monkey-patches `os.getpid()` to return different values and asserts the resulting directories differ:

```python

# From tests/test_per_run_workdir.py lines 38-48

def test_default_workdir_is_unique_per_process(self, monkeypatch):
    monkeypatch.setattr(os, 'getpid', lambda: 1000)
    dir1 = default_output_dir()
    monkeypatch.setattr(os, 'getpid', lambda: 2000)
    dir2 = default_output_dir()
    assert dir1 != dir2
    assert "1000" in str(dir1)
    assert "2000" in str(dir2)

```

2. **Sibling directory placement** — The new PID-based path is a **sibling** of the legacy fixed path (`$TMPDIR/book_skill_work`), not a child. This prevents legacy cleanup routines from deleting active run directories.

3. **Empty string handling** — Tests confirm that `BOOK_SKILL_WORKDIR=""` does not collapse to the current working directory.

## Practical Usage Examples

Access the configured directories in Python:

```python
from book_to_skill.config import OUTPUT_DIR, OUTPUT_TEXT, OUTPUT_META

print("Work directory :", OUTPUT_DIR)      # e.g., /tmp/book_skill_work-12345

print("Full text file :", OUTPUT_TEXT)     # /tmp/book_skill_work-12345/full_text.txt

print("Metadata file :", OUTPUT_META)      # /tmp/book_skill_work-12345/metadata.json

```

Run two extractions concurrently without conflict:

```python
import subprocess
import os

# Each subprocess gets its own PID and therefore its own work directory

subprocess.Popen(['python', '-m', 'book_to_skill', 'extract', 'book1.pdf'])
subprocess.Popen(['python', '-m', 'book_to_skill', 'extract', 'book2.pdf'])

```

## Summary

- **`default_output_dir()`** in [`book_to_skill/config.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/book_to_skill/config.py) embeds `os.getpid()` into the directory name
- **`OUTPUT_DIR`** resolves at import time, ensuring consistent paths throughout the process lifetime
- **`BOOK_SKILL_WORKDIR`** provides optional override with safe empty-string handling
- **Tests in [`tests/test_per_run_workdir.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/tests/test_per_run_workdir.py)** verify PID uniqueness, directory placement, and edge cases
- **Downstream files** ([`cli.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/cli.py), [`scripts/extract.py`](https://github.com/virgiliojr94/book-to-skill/blob/main/scripts/extract.py)) inherit isolation without additional code

## Frequently Asked Questions

### What happens if two extractions run at exactly the same time?

Each Python process receives a unique PID from the operating system. Since `default_output_dir()` includes this PID in the path, simultaneous extractions create directories like `/tmp/book_skill_work-12345` and `/tmp/book_skill_work-12346`—guaranteeing no file collisions.

### Can I use a fixed directory for all extractions?

Yes. Set `BOOK_SKILL_WORKDIR` to your preferred path. The code uses this value directly, bypassing the PID-based naming. Be aware that concurrent runs with a fixed directory will overwrite each other's files unless you implement your own locking mechanism.

### Why does the code use `or` instead of `.get()` with a default?

The expression `os.environ.get("BOOK_SKILL_WORKDIR") or default_output_dir()` ensures that an empty string environment variable evaluates as falsy and triggers the fallback. Using `.get()` with a default would return `""`, and `Path("")` resolves to the current working directory—an unsafe default that could pollute the project folder with temporary files.

### How does this design affect legacy cleanup scripts?

The PID-based directory (`book_skill_work-12345`) is a **sibling** of the old fixed path (`book_skill_work`), not a subdirectory. Legacy scripts that delete `book_skill_work` recursively cannot accidentally destroy active extraction directories, since they reside at the same level with different names.