Background Job Architecture for Codex Tasks: How the Codex Plugin Manages Detached Workers

Codex tasks launched with the --background flag run as detached Node workers managed through a lightweight job-tracking system that persists state to JSON files and streams logs to disk.

The openai/codex-plugin-cc repository implements a robust background job architecture that allows long-running Codex operations to execute without blocking the interactive Claude session. When you append --background to a task command, the system spawns an isolated worker process, assigns a unique job ID, and tracks the entire lifecycle through a dedicated state directory. This architecture supports task queuing, progress monitoring, cancellation, and result retrieval across multiple Claude sessions.

CLI Parsing and Worker Detachment

The entry point for background execution resides in scripts/codex-companion.mjs, which parses --background and --wait flags before delegating to the Bash sub-agent. When run_in_background: true is passed, the companion script instructs the sub-agent to detach the process rather than await completion.

# Launch a long-running task in the background

node scripts/codex-companion.mjs task --background "refactor the authentication module"

The worker immediately writes a job ID (formatted as task-<uid>) to stdout and returns control to the user. This non-blocking behavior allows the Claude session to continue while the worker executes the requested sub-command—whether task, review, or rescue—independently.

Job Persistence and File Structure

Each background job receives dedicated storage under the repository's state directory at .codex/state/jobs/. The state.mjs module handles file I/O through helpers like resolveJobFile and write, creating two artifacts per job:

• <job-id>.json: Contains metadata including status, timestamps, session ID, and execution context
• <job-id>.log: Streaming capture of the worker's stdout and stderr output

The system maintains a jobs index—a JSON list that tracks all historical jobs. To prevent unbounded growth, state.mjs implements automatic pruning with a configurable maximum (defaulting to 50 entries). When the cap is exceeded, the oldest entries and their associated log files are purged from disk, as verified in tests/state.test.mjs.

Session Scoping and Environment Isolation

Jobs are automatically tagged with the current Claude session ID via the SESSION_ID_ENV environment variable. The tracked-jobs.mjs module reads this variable to enforce session boundaries, ensuring that status and cancel commands only surface jobs belonging to the active session unless explicitly overridden.

This scoping mechanism prevents cross-contamination between different Claude windows or users while still allowing administrative oversight of all jobs when needed. The session ID is embedded in each job's JSON metadata during the initial write operation in state.mjs.

Lifecycle Management and Status Snapshots

The job-control.mjs module provides the core orchestration logic for interacting with persisted jobs. Its buildStatusSnapshot function (lines 13–40) categorizes jobs into three groups:

1. Running: Actively executing workers
2. Latest finished: Recently completed tasks
3. Recent: Historical jobs within the retention window

Each entry is enriched with a progress preview, elapsed time calculation, and an inferred human-readable phase such as "reviewing," "verifying," or "editing." This metadata assembly happens in real-time by reading the job JSON files and parsing the trailing content of the log files.

Cancellation and Result Retrieval

Two critical resolution functions handle job lifecycle transitions:

resolveCancelableJob (lines 81–108 in job-control.mjs): Locates active jobs (queued or running) for the current session or a specific job ID. Upon locating the target, the function signals termination and updates the job file's status to cancelled.

resolveResultJob (lines 56–78 in job-control.mjs): Identifies the most recent finished job (or a user-specified ID) and extracts the stored JSON output for return to the caller.

# Check progress of a specific job

node scripts/codex-companion.mjs status task-abc123 --wait --json

# Retrieve final results once completed

node scripts/codex-companion.mjs result task-abc123 --json

# Cancel an active background job

node scripts/codex-companion.mjs cancel task-abc123

HTTP Interface for UI Integration

While the CLI provides direct control, app-server.mjs exposes HTTP endpoints (/codex:status, /codex:cancel, etc.) that allow the Claude UI to interact with the job system programmatically. This server layer translates HTTP requests into the same job-control.mjs functions used by the command-line interface, ensuring consistent behavior across interaction modes.

Summary

• Detached execution: The --background flag triggers run_in_background: true in codex-companion.mjs, spawning independent Node workers that return job IDs immediately.
• Dual-file persistence: Each job creates a JSON metadata file and a .log stream in .codex/state/jobs/, managed by state.mjs with automatic pruning (default limit: 50 jobs).
• Session awareness: tracked-jobs.mjs filters jobs by SESSION_ID_ENV to isolate work between Claude sessions.
• Rich status reporting: buildStatusSnapshot in job-control.mjs categorizes jobs and infers execution phases from log content.
• Complete lifecycle control: The architecture supports cancellation via resolveCancelableJob and result retrieval via resolveResultJob, accessible through both CLI and HTTP endpoints.

Frequently Asked Questions

How does the Codex plugin handle process detachment when running background tasks?

The plugin uses the Bash sub-agent implementation in process.mjs to spawn Node workers with detached process flags. When codex-companion.mjs detects --background, it passes run_in_background: true to the sub-agent, which creates an independent process group. The worker writes its job ID to stdout and disassociates from the parent terminal, allowing the Claude session to continue while execution proceeds asynchronously.

Where are background job logs and metadata stored?

Job artifacts persist in .codex/state/jobs/ within the repository root. Each job receives a unique ID (e.g., task-abc123) and generates two files: a .json file containing structured metadata (status, timestamps, session ID) and a .log file capturing real-time stdout/stderr streams. The state.mjs module handles all file operations, including atomic writes and index maintenance.

What happens when the maximum number of background jobs is reached?

When the jobs index exceeds the configurable limit (default 50), state.mjs automatically prunes the oldest entries. This cleanup removes both the JSON metadata and associated log files from disk, preventing storage bloat while retaining recent history. The pruning logic is tested in tests/state.test.mjs at line 43.

Can I cancel a background job started in a different Claude session?

By default, no. The tracked-jobs.mjs module restricts cancellation to jobs tagged with the current SESSION_ID_ENV value. However, if you specify the exact job ID (e.g., cancel task-abc123), resolveCancelableJob in job-control.mjs can locate and terminate jobs across sessions, provided you have the correct identifier and sufficient permissions to access the state directory.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}