How Background Jobs Work in the Codex Plugin: Lifecycle and Implementation

Background jobs in the Codex plugin run as detached Node.js child processes that persist state to JSON files in the user's temporary directory, enabling asynchronous execution of long-running tasks with real-time progress monitoring.

The openai/codex-plugin-cc repository implements a sophisticated job execution system for handling resource-intensive operations like code reviews and rescue tasks. When invoked with the --background flag, the plugin orchestrates stateful, detached workers that survive parent process termination. This architecture separates job management from execution through persistent state files and per-job log streams.

State Management Architecture

The background job system relies on a dual-layer persistence strategy that tracks metadata and execution logs independently.

Persistent Job Records in state.mjs

The foundation of the background job system lies in plugins/codex/scripts/lib/state.mjs, which manages a JSON-based state directory under the user's temporary folder. This module provides atomic operations for loading, saving, pruning, and upserting job metadata via functions like upsertJob. Each job record tracks critical fields including status, timestamps, PID references, and log file paths, enabling recovery even after the parent process exits.

Progress Tracking via tracked-jobs.mjs

Per-job observability is handled by plugins/codex/scripts/lib/tracked-jobs.mjs. The createTrackedProgress function initializes dedicated log files for each job, while appendLogLine and appendLogBlock provide thread-safe progress streaming. This module also defines runTrackedJob, which binds a progress reporter to the job's log file and manages the transition from active execution to terminal states.

Enqueuing and Spawning Background Tasks

When users request background execution through the --background or --wait flags, the companion script invokes enqueueBackgroundTask in plugins/codex/scripts/codex-companion.mjs. This function orchestrates four critical steps:

1. Calls createTrackedProgress to generate a unique log file.
2. Writes an initial "Queued for background execution" entry.
3. Spawns a detached Node child process via spawnDetachedTaskWorker.
4. Persists a "queued" job record containing the child PID and log path using writeJobFile and upsertJob.

The spawnDetachedTaskWorker function creates an isolated execution environment by spawning the same script with the task-worker subcommand and critical Node.js options:

// Spawn the detached worker (platform‑agnostic)
function spawnDetachedTaskWorker(cwd, jobId) {
  const script = path.join(ROOT_DIR, "scripts", "codex-companion.mjs");
  const child = spawn(process.execPath, [script, "task-worker", "--cwd", cwd, "--job-id", jobId], {
    cwd,
    env: process.env,
    detached: true,
    stdio: "ignore",
    windowsHide: true,
  });
  child.unref();
  return child;
}

The detached: true and stdio: "ignore" options ensure the child process can outlive its parent, while child.unref() allows the parent event loop to exit without waiting for the worker.

The Background Job Lifecycle

Once detached, the worker process manages its own lifecycle while updating shared state for external monitoring.

Worker Initialization and Execution

The child process executes the task-worker subcommand within codex-companion.mjs. The worker immediately invokes runTrackedJob from tracked-jobs.mjs, which accepts a runner function and a progress reporter bound to the job's log file. This design ensures all stdout, stderr, and custom telemetry streams into the persistent log for later inspection.

// Detached worker runs the job and records results
import { runTrackedJob } from "./tracked-jobs.mjs";

async function workerMain(job, runner) {
  // `runner` receives a progress reporter bound to the job’s log file
  const execResult = await runTrackedJob(job, () => runner(progress), { logFile: job.logFile });
  // execResult contains { exitStatus, payload, rendered, ... }
}

State Transitions and Completion

Throughout execution, runTrackedJob mutates the job record to reflect current reality. Upon completion, it atomically updates the status to completed or failed, records final timestamps, captures output payloads, and clears the PID field. The plugins/codex/scripts/lib/job-control.mjs module provides the /codex:status command implementation, using enrichJob to read log previews, infer current phases, and calculate elapsed times from the persistent state.

// Enqueue a background task (simplified)
import { createTrackedProgress, appendLogLine, writeJobFile, upsertJob } from "./tracked-jobs.mjs";
import { spawnDetachedTaskWorker } from "./codex-companion.mjs";

function enqueueBackgroundTask(cwd, job) {
  const { logFile } = createTrackedProgress(job);
  appendLogLine(logFile, "Queued for background execution.");

  const child = spawnDetachedTaskWorker(cwd, job.id);
  const queued = { ...job, status: "queued", phase: "queued", pid: child.pid, logFile };
  writeJobFile(job.workspaceRoot, job.id, queued);
  upsertJob(job.workspaceRoot, queued);
  return queued;
}

Summary

• Background jobs persist metadata in JSON files managed by state.mjs within the user's temporary directory.
• The enqueueBackgroundTask function spawns detached Node processes via spawnDetachedTaskWorker with detached: true and stdio: "ignore".
• Workers execute the task-worker subcommand and report progress through runTrackedJob in tracked-jobs.mjs.
• Job status flows through queued, active, and terminal (completed/failed) states, with PIDs cleared upon completion.
• The job-control.mjs module enables status queries via /codex:status by reading persistent logs and enriching job records.

Frequently Asked Questions

Where does the Codex plugin store background job state?

According to the source code in plugins/codex/scripts/lib/state.mjs, job metadata is persisted as JSON files in a state directory located under the user's temporary folder. This includes status flags, timestamps, PIDs, and references to per-job log files generated by createTrackedProgress.

How does the parent CLI exit while the background job continues running?

The spawnDetachedTaskWorker function in codex-companion.mjs spawns the worker with detached: true and stdio: "ignore" options, then calls child.unref(). This combination decouples the child from the parent's event loop, allowing the parent process to terminate without killing the background worker.

What happens if a background job fails or crashes?

When runTrackedJob catches an error or non-zero exit code, it updates the job record status to failed, persists the error output to the log file via appendLogBlock, clears the PID field, and records the completion timestamp. The persistent state remains available for inspection via /codex:status or direct log reading.

How can I monitor the progress of a running background job?

The /codex:status command leverages job-control.mjs to read the per-job log file via enrichJob, which parses the latest log entries and calculates elapsed time. Users can query this status anytime, even if the original parent process has exited, because the state resides in the temporary directory's JSON files.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}