# How cleanup_background_hooks Handles Stuck Processes and Process Groups in abx-dl

> Learn how cleanup_background_hooks in abx-dl safely terminates stuck processes and process groups using SIGTERM and SIGKILL, preventing accidental system process termination.

- Repository: [ArchiveBox/abx-dl](https://github.com/archivebox/abx-dl)
- Tags: internals
- Published: 2026-02-25

---

**The `cleanup_background_hooks` function in [`abx_dl/executor.py`](https://github.com/archivebox/abx-dl/blob/main/abx_dl/executor.py) terminates background hook processes using a multi-step algorithm that sends SIGTERM to process groups, waits, then escalates to SIGKILL for stubborn processes, while validating PID files to prevent accidental termination of unrelated system processes.**

The `cleanup_background_hooks` function is a critical component of the [archivebox/abx-dl](https://github.com/archivebox/abx-dl) repository that ensures background hook processes—such as Chrome-based crawlers—are properly terminated after a download completes. Located in [`abx_dl/executor.py`](https://github.com/archivebox/abx-dl/blob/main/abx_dl/executor.py) (lines 204–210 and 226–288), this function implements a robust cleanup strategy that handles everything from graceful exits to kernel-level unkillable processes.

## How cleanup_background_hooks Terminates Background Processes

The function implements a four-step termination sequence that progressively escalates from polite requests to forceful kills, while protecting against PID reuse and stale lock files.

### Step 1: Validating PID Files to Prevent Accidental Kills

Before sending any signals, `cleanup_background_hooks` calls `validate_pid_file` from [`abx_dl/process_utils.py`](https://github.com/archivebox/abx-dl/blob/main/abx_dl/process_utils.py) (lines 28–55) to verify that the PID file actually belongs to the hook that created it. This validation matches the file's modification time to the process start time and optionally checks the command line. If validation fails—indicating PID reuse or a stale file—the function simply deletes the PID file and marks the hook as successfully cleaned, avoiding accidental termination of unrelated system processes.

### Step 2: Graceful Termination with SIGTERM

For valid processes, the function first attempts graceful shutdown by sending `SIGTERM` to the entire process group using `os.killpg(pid, signal.SIGTERM)`. This approach, implemented around lines 226–250 of [`executor.py`](https://github.com/archivebox/abx-dl/blob/main/executor.py), ensures that detached child processes—such as Chrome tabs spawned by the hook—receive the termination signal alongside the parent. If the process group kill fails, the function falls back to `os.kill(pid, signal.SIGTERM)` targeting the individual PID.

After sending SIGTERM, the function waits approximately 2 seconds and checks `is_process_alive` (from [`process_utils.py`](https://github.com/archivebox/abx-dl/blob/main/process_utils.py), lines 95–101) to determine if the process has exited.

### Step 3: Forceful Termination with SIGKILL

If the process remains alive after the SIGTERM grace period, `cleanup_background_hooks` escalates to forceful termination. It sends `SIGKILL` to the entire process group via `os.killpg(pid, signal.SIGKILL)`, which the operating system handles immediately without allowing the process to catch or ignore the signal. As with the graceful phase, if the group-level kill fails, the function falls back to killing the single PID.

Following the SIGKILL, the function waits approximately 1 second before checking again with `is_process_alive`.

### Step 4: Handling Unkillable Processes

In rare cases—particularly on macOS where processes can enter a kernel-level "UN*" (uninterruptible) state—a process may survive even SIGKILL. After the forceful termination attempt, `cleanup_background_hooks` performs a final check. If `is_process_alive` still returns `True`, the function logs a warning to stderr (only when running in a TTY) and marks the hook as failed with a "Process unkillable" error. Crucially, this does not block the overall cleanup operation—the function continues processing remaining hooks and exits, leaving the zombie process to persist until the next system reboot.

## Why Process Group Killing Matters for Detached Children

Background hooks in abx-dl frequently spawn detached subprocesses. For example, Chrome-based hooks launch browser instances that may fork additional processes for tabs, GPU rendering, or extensions. If `cleanup_background_hooks` only killed the parent PID, these child processes would become orphaned and continue consuming system resources indefinitely.

By using `os.killpg` to target the entire process group—mirroring how a shell's `Ctrl-C` terminates a pipeline—the function ensures that all descendants receive the termination signal simultaneously. This approach is implemented in the SIGTERM and SIGKILL phases described above, with fallback logic to handle edge cases where the process group may no longer exist.

## Code Examples

### Manually Invoking Cleanup After a Crawl

You can trigger the cleanup routine manually to terminate any lingering background hooks:

```python
from pathlib import Path
from abx_dl.executor import cleanup_background_hooks

output_dir = Path("/tmp/abx-dl-output/2024-01-01/example.com")
index_path = output_dir / "index.jsonl"
is_tty = True   # prints warnings to stderr when needed

# Run the cleanup – it will terminate any lingering background hooks

cleanup_background_hooks(output_dir, index_path, is_tty)

```

The function scans `output_dir` for `on_*.pid` files, validates them using `validate_pid_file`, and applies the multi-step termination logic.

### Simulating a Stuck Chrome-Based Hook

This example demonstrates how the function handles a process group containing detached Chrome processes:

```python
import os
import signal
import time
from pathlib import Path
from abx_dl.process_utils import write_pid_file_with_mtime

# Pretend a Chrome hook started a process group with PID 12345

pid = 12345
pid_file = Path("/tmp/abx-dl-output/example/on_ChromeHook.pid")
write_pid_file_with_mtime(pid_file, pid, time.time())

# Later, during cleanup, the function will attempt:

#   os.killpg(12345, signal.SIGTERM)   # graceful group kill

#   os.killpg(12345, signal.SIGKILL)  # forceful group kill

# If the process remains, it will emit a warning.

```

### Detecting Stale PID Files

Use the validation utility to prevent accidental termination of unrelated processes:

```python
from abx_dl.process_utils import validate_pid_file
from pathlib import Path

pid_file = Path("/tmp/abx-dl-output/example/on_OldHook.pid")
cmd_file = pid_file.with_suffix('.sh')

if not validate_pid_file(pid_file, cmd_file):
    print("Stale PID – nothing to kill")
else:
    print("Valid PID – cleanup will proceed")

```

## Key Implementation Files

| File | Relevant Function / Section | Description |
|------|----------------------------|-------------|
| **[`abx_dl/executor.py`](https://github.com/archivebox/abx-dl/blob/main/abx_dl/executor.py)** | `cleanup_background_hooks` (lines 204–210, 226–288) | Main routine that scans for `.pid` files, validates them, and performs the multi-step termination of background hooks. |
| **[`abx_dl/process_utils.py`](https://github.com/archivebox/abx-dl/blob/main/abx_dl/process_utils.py)** | `validate_pid_file` (lines 28–55) | Verifies that a PID file belongs to the expected process using mtime and optional command-line verification. |
| | `is_process_alive` (lines 95–101) | Checks process existence after signaling to determine if further action is needed. |
| **[`abx_dl/executor.py`](https://github.com/archivebox/abx-dl/blob/main/abx_dl/executor.py)** | `_finalize_background_hook` (lines 94–138) | Writes the hook's final JSONL record to `index.jsonl` and cleans up log files after termination. |

## Summary

- **`cleanup_background_hooks`** implements a four-step termination sequence: validation, SIGTERM, SIGKILL, and unkillable detection.
- **Process group killing** via `os.killpg` ensures detached children (like Chrome processes) are terminated alongside parent hooks.
- **PID validation** prevents accidental termination of unrelated processes by verifying file ownership before signaling.
- **Graceful degradation** allows the cleanup to continue even when processes enter uninterruptible kernel states on macOS.

## Frequently Asked Questions

### What is the difference between SIGTERM and SIGKILL in cleanup_background_hooks?

**SIGTERM** is sent first as a graceful shutdown request, allowing the hook process to clean up temporary files and flush buffers. The function waits approximately 2 seconds after sending SIGTERM to the process group. If the process remains alive, **SIGKILL** is sent, which the operating system handles immediately without allowing the process to catch or ignore the signal, guaranteeing termination except in uninterruptible kernel states.

### How does cleanup_background_hooks prevent killing unrelated system processes?

Before sending any signals, the function calls `validate_pid_file` from [`abx_dl/process_utils.py`](https://github.com/archivebox/abx-dl/blob/main/abx_dl/process_utils.py) to verify the PID file belongs to the hook that created it. This validation matches the file's modification time to the process start time and optionally checks the command line. If validation fails—indicating PID reuse or a stale file—the function deletes the PID file and marks the hook as successfully cleaned without sending any signals, preventing accidental termination of unrelated processes.

### Why does cleanup_background_hooks use process groups instead of individual PIDs?

Background hooks frequently spawn detached subprocesses, such as Chrome browser instances that fork additional processes for tabs, GPU rendering, and extensions. Killing only the parent PID would orphan these children, leaving them to consume system resources indefinitely. By using `os.killpg` to target the entire process group, the function ensures all descendants receive the termination signal simultaneously, mirroring how a shell's Ctrl-C terminates an entire pipeline.

### What happens if a process enters an unkillable state on macOS?

If a process survives the SIGKILL attempt—possible on macOS when processes enter a kernel-level "UN*" (uninterruptible) state—the function performs a final check with `is_process_alive`. If the process remains alive, it logs a warning to stderr (only when running in a TTY) and marks the hook as failed with a "Process unkillable" error. Crucially, this does not block the overall cleanup operation; the function continues processing remaining hooks and exits, leaving the zombie process to persist until the next system reboot.