How Background Job Execution Works in the OpenAI Codex Plugin

The Codex plugin implements background job execution by persisting job metadata to JSON state files, spawning detached Node.js child processes via spawnDetachedTaskWorker, and using a dedicated task-worker subcommand to run tasks asynchronously while allowing the parent process to exit.

The openai/codex-plugin-cc repository provides a robust asynchronous job system for long-running Codex operations such as code reviews and rescue tasks. Background job execution enables users to launch intensive workloads that persist beyond the terminal session, with full support for progress tracking, status queries, and log retrieval. The architecture splits responsibility across four core modules that handle state persistence, process isolation, and lifecycle management.

Architecture Overview

The background job system follows a clear separation between the foreground CLI and background workers:

1. State Management: Job metadata lives in JSON files within a temporary state directory.
2. Enqueueing: The companion script creates job records and spawns detached workers when the --background flag is present.
3. Execution: A detached Node process runs the actual task and updates the shared state.
4. Monitoring: Status commands read the persistent state to display progress without blocking the terminal.

Persisting Job State with state.mjs

All job metadata is stored in a JSON-based state directory under the user’s temporary folder. The state.mjs module provides the low-level primitives for job persistence, including loadJob, saveJob, upsertJob, and pruneJobs.

Key responsibilities include:

• Atomic writes: Ensures job records (containing status, timestamps, PID, and log file paths) are written safely to disk.
• Job indexing: Maintains an index of active jobs per workspace for fast lookups.
• Cleanup: Removes stale entries when jobs complete or are cancelled.

When a user enqueues a background task, the system immediately writes a "queued" status record via writeJobFile and upsertJob, ensuring the job survives even if the spawn operation is interrupted.

Tracking Progress and Logs

The tracked-jobs.mjs module handles the creation and management of per-job log files and execution wrappers. This module defines three critical functions:

• createTrackedProgress(job): Initializes a new log file for the job and returns a logFile path.
• appendLogLine(logFile, message) and appendLogBlock(logFile, data): Stream progress updates and structured output to the log.
• runTrackedJob(job, runnerFn, options): Wraps the actual task execution, binding a progress reporter to the job’s log file and handling status transitions.

When the worker process begins execution, it calls runTrackedJob, which updates the job status from "queued" to "running" and eventually to "completed" or "failed" based on the exit code.

Enqueuing Background Tasks

The entry point for background execution is codex-companion.mjs, which implements the enqueueBackgroundTask function. When the user supplies the --background or --wait flags, this function orchestrates the handoff from foreground to background:

// From codex-companion.mjs
function enqueueBackgroundTask(cwd, job) {
  const { logFile } = createTrackedProgress(job);
  appendLogLine(logFile, "Queued for background execution.");

  const child = spawnDetachedTaskWorker(cwd, job.id);
  const queued = {
    ...job,
    status: "queued",
    phase: "queued",
    pid: child.pid,
    logFile
  };
  writeJobFile(job.workspaceRoot, job.id, queued);
  upsertJob(job.workspaceRoot, queued);
  return queued;
}

This sequence ensures that:

• A log file exists before the worker starts.
• The job record contains the child PID for later signalling.
• The parent can return immediately while the worker initializes.

The Detached Worker Process

Process isolation is achieved through spawnDetachedTaskWorker, which launches a new Node.js process with specific options to ensure true background execution:

function spawnDetachedTaskWorker(cwd, jobId) {
  const script = path.join(ROOT_DIR, "scripts", "codex-companion.mjs");
  const child = spawn(process.execPath, [
    script,
    "task-worker",
    "--cwd", cwd,
    "--job-id", jobId
  ], {
    cwd,
    env: process.env,
    detached: true,
    stdio: "ignore",
    windowsHide: true,
  });
  child.unref();
  return child;
}

Critical options include:

• detached: true: Creates a new process group, preventing SIGINT from the parent terminal from reaching the child.
• stdio: "ignore": Disconnects the child from the parent’s stdin/stdout/stderr.
• child.unref(): Allows the parent event loop to exit even though the child is running.

The child process executes the task-worker subcommand, which loads the job state via loadJob and invokes runTrackedJob to perform the actual work.

Job Lifecycle and Status Monitoring

Once detached, the worker process manages the full job lifecycle through the shared state system. As the task progresses:

1. Phase updates: The worker writes intermediate phases (e.g., "analyzing", "executing") to the log and updates the job record.
2. Completion: Upon finishing, runTrackedJob records the exit status, output payload, rendered results, and timestamps.
3. Cleanup: The PID is cleared from the record to indicate the process has ended, though the log file persists for inspection.

The job-control.mjs module provides the read-path for status queries. Its enrichJob function reads the log preview, infers the current phase from log content, and calculates elapsed times. This powers the /codex:status command, allowing users to query background job progress without blocking.

Summary

• State persistence: Job metadata lives in JSON files managed by state.mjs, ensuring durability across process restarts.
• Process isolation: spawnDetachedTaskWorker creates truly independent Node.js processes using detached: true and stdio: "ignore".
• Progress tracking: tracked-jobs.mjs provides runTrackedJob for structured logging and status updates throughout the job lifecycle.
• Async handoff: enqueueBackgroundTask writes the job record, spawns the worker, and returns immediately, enabling non-blocking CLI usage.
• Status queries: job-control.mjs reconstructs job status from persistent state and log files for the /codex:status command.

Frequently Asked Questions

How does the Codex plugin detach a background job from the parent terminal?

The plugin uses Node.js spawn with detached: true and stdio: "ignore" in spawnDetachedTaskWorker, followed by child.unref(). This combination creates a new process group, disconnects all stdio streams, and removes the child from the parent’s reference count, allowing the parent to exit while the worker continues running.

Where are background job logs and state stored?

Job state is persisted as JSON files in a temporary directory under the user’s system temp folder, managed by state.mjs. Each job also receives a dedicated log file created by createTrackedProgress in tracked-jobs.mjs, which stores all progress updates, command output, and error messages.

How can I check the status of a running background job?

Use the /codex:status command, which invokes functions in job-control.mjs to read the persistent job state. The enrichJob function parses the log file, infers the current phase (queued, running, analyzing, completed, or failed), and calculates elapsed time without requiring the worker process to be active or responsive to signals.

What happens if the parent process exits while a background job is running?

The job continues executing because the worker process is fully detached. The worker updates the same JSON state files and log files independently. When the parent restarts, it can read the current status from state.mjs and retrieve results from the job’s log file, even though the original parent process no longer exists.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}