How the Codex Plugin Manages Background Jobs: Architecture and Implementation

The Codex plugin manages background jobs by persisting job metadata to JSON state files, spawning detached Node processes via spawnDetachedTaskWorker, and using helper functions in tracked-jobs.mjs to update progress logs and status throughout the job lifecycle.

The OpenAI Codex plugin for command-line coding tasks provides a robust asynchronous job system that allows long-running operations like code reviews and rescues to execute without blocking the terminal. When users invoke commands with the --background flag, the plugin leverages a specialized architecture involving state persistence, detached process spawning, and log-based progress tracking. This implementation ensures that background jobs survive parent process termination while remaining queryable through the /codex:status command.

Anatomy of the Background Job System

The background job implementation spans four core modules within the openai/codex-plugin-cc repository. Each module handles a distinct responsibility: state persistence, job tracking, process lifecycle management, and runtime control.

State Management with state.mjs

All job metadata—including status, timestamps, PID, and log file paths—is persisted in a JSON-based state directory under the user’s temporary folder. The loadState, saveState, upsertJob, and prune utilities in plugins/codex/scripts/lib/state.mjs handle atomic read/write operations for job records. This module ensures that job state survives process restarts and allows the job-control.mjs module to reconstruct job status at any time.

Job Tracking Utilities in tracked-jobs.mjs

The plugins/codex/scripts/lib/tracked-jobs.mjs module provides the bridge between job state and execution. Key functions include:

• createTrackedProgress(job): Initializes a dedicated log file for the job and returns its path.
• appendLogLine(logFile, message): Writes timestamped progress updates to the job’s log.
• runTrackedJob(job, runner, options): Wraps the actual task execution, captures output, and updates the job record to completed or failed upon finish.

These utilities ensure that every background job maintains an auditable log trail and consistent state transitions.

Enqueueing Background Tasks

When a user supplies the --background or --wait flags, the companion script triggers the background execution pipeline defined in codex-companion.mjs.

The enqueueBackgroundTask Function

The enqueueBackgroundTask function (lines 84-99 in plugins/codex/scripts/codex-companion.mjs) orchestrates the handoff from foreground to background:

1. Calls createTrackedProgress to generate a unique log file.
2. Writes an initial "Queued for background execution." entry.
3. Invokes spawnDetachedTaskWorker to create the worker process.
4. Persists a "queued" job record containing the child PID and log file path via writeJobFile and upsertJob.

// Simplified enqueueing logic from codex-companion.mjs
import { createTrackedProgress, appendLogLine, writeJobFile, upsertJob } from "./lib/tracked-jobs.mjs";
import { spawnDetachedTaskWorker } from "./codex-companion.mjs";

function enqueueBackgroundTask(cwd, job) {
  const { logFile } = createTrackedProgress(job);
  appendLogLine(logFile, "Queued for background execution.");

  const child = spawnDetachedTaskWorker(cwd, job.id);
  const queued = { 
    ...job, 
    status: "queued", 
    phase: "queued", 
    pid: child.pid, 
    logFile 
  };
  
  writeJobFile(job.workspaceRoot, job.id, queued);
  upsertJob(job.workspaceRoot, queued);
  return queued;
}

Spawning Detached Workers

The spawnDetachedTaskWorker function (lines 71-80) creates a platform-agnostic detached process:

import { spawn } from "child_process";
import path from "path";

function spawnDetachedTaskWorker(cwd, jobId) {
  const script = path.join(ROOT_DIR, "scripts", "codex-companion.mjs");
  const child = spawn(process.execPath, [
    script, 
    "task-worker", 
    "--cwd", cwd, 
    "--job-id", jobId
  ], {
    cwd,
    env: process.env,
    detached: true,
    stdio: "ignore",
    windowsHide: true,
  });
  
  child.unref();
  return child;
}

The detached: true and stdio: "ignore" options ensure the worker process continues running after the parent exits, while windowsHide: true prevents console window flicker on Windows systems.

The Detached Worker Process

Once spawned, the detached worker operates independently, executing the actual task and updating shared state.

task-worker Subcommand Execution

The child process runs codex-companion.mjs with the task-worker subcommand. This entry point receives the --job-id and --cwd arguments, reconstructs the job context from the state directory, and invokes the appropriate runner. Because the process is detached, it maintains its own event loop and handles process signals independently of the initiating terminal session.

Job Lifecycle and Progress Updates

During execution, the worker calls runTrackedJob (defined in tracked-jobs.mjs, lines 42-104) to wrap the task logic:

import { runTrackedJob } from "./lib/tracked-jobs.mjs";

async function workerMain(job, runner) {
  // runner receives a progress reporter bound to the job's log file
  const execResult = await runTrackedJob(job, () => runner(progress), { 
    logFile: job.logFile 
  });
  
  // execResult contains { exitStatus, payload, rendered, ... }
  // Status is automatically updated to 'completed' or 'failed'
}

The runTrackedJob function handles:

• Writing start timestamps.
• Streaming output to the log file via appendLogBlock.
• Capturing exit codes and error payloads.
• Updating the job record status to completed or failed upon termination.
• Clearing the PID field to indicate the process no longer exists.

Querying and Controlling Background Jobs

While jobs run independently, users can monitor them through the /codex:status command. The plugins/codex/scripts/lib/job-control.mjs module (specifically the enrichJob function at lines 61-80) reads the persistent state, parses the log file preview, infers the current execution phase, and calculates elapsed times. This allows the CLI to display real-time progress without requiring inter-process communication—instead relying on the shared JSON state and log files.

Summary

• State persistence: Job metadata lives in JSON files managed by state.mjs, ensuring durability across process boundaries.
• Process isolation: Background jobs run as detached Node processes via spawnDetachedTaskWorker, allowing the parent CLI to exit immediately.
• Log-centric tracking: tracked-jobs.mjs maintains per-job log files and status records, enabling asynchronous progress monitoring.
• Lifecycle management: The runTrackedJob wrapper handles execution, error capture, and final state updates automatically.
• Status querying: job-control.mjs reconstructs job status from persisted state without requiring active connections to worker processes.

Frequently Asked Questions

How does the Codex plugin ensure background jobs survive terminal closure?

The plugin uses Node.js spawn with detached: true and stdio: "ignore" in spawnDetachedTaskWorker. This creates a new process group independent of the parent shell, allowing the worker to continue executing even when the initiating terminal session closes or the parent process exits.

Where does the Codex plugin store background job state and logs?

Job state is stored as JSON files in a temporary directory under the user's system temp folder, managed by plugins/codex/scripts/lib/state.mjs. Individual job logs are created by createTrackedProgress in tracked-jobs.mjs and written to via appendLogLine and appendLogBlock during execution.

Can I check the status of a running background job without interrupting it?

Yes. The /codex:status command uses job-control.mjs to read the persisted job state and log files. The enrichJob function parses this data to display current phase, elapsed time, and log previews without requiring inter-process communication or interrupting the detached worker process.

What happens if a background job fails or crashes?

The runTrackedJob function in tracked-jobs.mjs captures exit codes and exceptions, updating the job record status to failed and recording error details in the log file. The PID is cleared from state upon completion regardless of success or failure, allowing job-control.mjs to determine the process has ended when next queried.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}