Background Job Execution Architecture in the OpenAI Codex Plugin

The Codex plugin implements background job execution through a detached Node.js child process system that persists job state to JSON files, enabling asynchronous task running with status tracking and log management.

The openai/codex-plugin-cc repository provides a robust system for executing long-running tasks—such as code review, rescue operations, and generic jobs—outside the main process lifecycle. This architecture ensures that users can enqueue work, disconnect from the terminal, and later query progress or retrieve results through persistent state management.

Core Architectural Components

The background job system is modularized across four primary modules that handle state persistence, job tracking, enqueueing logic, and lifecycle management.

State Persistence Layer

All job metadata is stored in a JSON-based state directory located within the user’s temporary folder. The state.mjs module (plugins/codex/scripts/lib/state.mjs) provides the foundational CRUD operations for this persistence layer, including load, save, prune, and upsert functions for job records. Each job entry tracks critical fields including status, timestamps, process ID (PID), and log file paths.

Job Tracking Infrastructure

The tracked-jobs.mjs module (plugins/codex/scripts/lib/tracked-jobs.mjs) defines the utilities for creating log files, constructing job records, and updating progress during execution. The runTrackedJob function (lines 42–104) serves as the primary execution wrapper, binding a progress reporter to the job’s log file and handling the transition from queued to completed or failed states.

Task Enqueueing and Worker Spawning

The codex-companion.mjs script (plugins/codex/scripts/codex-companion.mjs) acts as the CLI entry point and orchestrator. When the user supplies the --background or --wait flags, the script invokes enqueueBackgroundTask (lines 84–99), which coordinates the handoff to the background execution layer.

The Detached Worker Pattern

The architecture leverages Node.js detached processes to achieve true background execution isolation.

Spawn Mechanics

The spawnDetachedTaskWorker function (lines 71–80 in codex-companion.mjs) creates a detached child process using Node’s spawn with specific configuration flags:

function spawnDetachedTaskWorker(cwd, jobId) {
  const script = path.join(ROOT_DIR, "scripts", "codex-companion.mjs");
  const child = spawn(process.execPath, [script, "task-worker", "--cwd", cwd, "--job-id", jobId], {
    cwd,
    env: process.env,
    detached: true,
    stdio: "ignore",
    windowsHide: true,
  });
  child.unref();
  return child;
}

The detached: true and stdio: "ignore" options ensure the parent process can exit while the worker continues execution. The child.unref() call prevents the parent from waiting for the child to exit before terminating itself.

Worker Execution Flow

The detached child runs the same codex-companion.mjs script with the task-worker sub-command. This worker process:

1. Loads the job definition from the persistent state store
2. Invokes runTrackedJob to execute the actual command
3. Updates the same job record with exit status, output payload, and rendered results
4. Clears the PID field upon completion to indicate the process is no longer active

Job Lifecycle Management

The transition from enqueueing to completion follows a strict state machine managed through the coordination of multiple modules.

Enqueueing Sequence

When enqueueBackgroundTask is called, the system executes the following sequence:

function enqueueBackgroundTask(cwd, job) {
  const { logFile } = createTrackedProgress(job);
  appendLogLine(logFile, "Queued for background execution.");

  const child = spawnDetachedTaskWorker(cwd, job.id);
  const queued = { 
    ...job, 
    status: "queued", 
    phase: "queued", 
    pid: child.pid, 
    logFile 
  };
  writeJobFile(job.workspaceRoot, job.id, queued);
  upsertJob(job.workspaceRoot, queued);
  return queued;
}

This creates a persistent log file, writes an initial status message, spawns the detached worker, and persists the job record with the child’s PID.

Progress Tracking and Completion

The worker updates job state through appendLogLine and appendLogBlock for incremental progress updates. Upon completion, runTrackedJob atomically updates the status to completed or failed, records timestamps, and clears the PID field.

The job-control.mjs module (plugins/codex/scripts/lib/job-control.mjs) provides the enrichJob function (lines 61–80) for status queries, reading log previews, inferring the current execution phase, and formatting elapsed times for the /codex:status command.

Implementation Example

The following example demonstrates the complete flow from enqueueing to execution:

// Enqueue a background task
import { createTrackedProgress, appendLogLine, writeJobFile, upsertJob } from "./tracked-jobs.mjs";
import { spawnDetachedTaskWorker } from "./codex-companion.mjs";

function enqueueBackgroundTask(cwd, job) {
  const { logFile } = createTrackedProgress(job);
  appendLogLine(logFile, "Queued for background execution.");

  const child = spawnDetachedTaskWorker(cwd, job.id);
  const queued = { ...job, status: "queued", phase: "queued", pid: child.pid, logFile };
  writeJobFile(job.workspaceRoot, job.id, queued);
  upsertJob(job.workspaceRoot, queued);
  return queued;
}

// Detached worker execution
import { runTrackedJob } from "./tracked-jobs.mjs";

async function workerMain(job, runner) {
  const execResult = await runTrackedJob(
    job, 
    () => runner(progress), 
    { logFile: job.logFile }
  );
  // execResult contains { exitStatus, payload, rendered, ... }
}

Summary

• State Persistence: Job metadata is stored as JSON in the user’s temporary directory via state.mjs, enabling recovery across process restarts.
• Process Isolation: Background jobs run as detached Node.js child processes spawned through spawnDetachedTaskWorker, allowing the parent to exit immediately after enqueueing.
• Progress Tracking: The tracked-jobs.mjs module provides createTrackedProgress and runTrackedJob for log file management and atomic status updates.
• Lifecycle Management: The job-control.mjs module enables status queries and phase inference through the enrichJob function, supporting the /codex:status command interface.

Frequently Asked Questions

How does the Codex plugin isolate background jobs from the main process?

The plugin uses Node.js spawn with detached: true and stdio: "ignore" options in the spawnDetachedTaskWorker function. This creates a separate process group that survives parent termination, while child.unref() ensures the parent process can exit without waiting for the worker to complete.

Where is job state persisted in the Codex plugin architecture?

Job state is persisted in a JSON-based directory structure under the user’s temporary folder, managed by the state.mjs module. Each job record includes status, timestamps, PID, and log file paths, enabling asynchronous status queries through the upsertJob and writeJobFile functions.

How can I check the status of a running background job?

Query the job state using the job-control.mjs module, specifically the enrichJob function (lines 61–80), which reads the persistent log file, infers the current execution phase, and calculates elapsed time. This powers the /codex:status command interface.

What happens when a background job completes or fails?

The runTrackedJob function in tracked-jobs.mjs atomically updates the job record status to completed or failed, records the exit timestamp and output payload, and clears the PID field. Results remain accessible in the log file and JSON state for subsequent retrieval.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}