How the Background Job Execution and Tracking System Works in the Codex Plugin

The Codex plugin implements a persistent background job system using detached Node.js processes, JSON state files in the user's temporary directory, and per-job log files to track progress across review, rescue, and generic tasks.

The openai/codex-plugin-cc repository provides a robust mechanism for executing long-running operations—such as code reviews and rescue tasks—outside the foreground CLI session. This background job execution and tracking system allows users to enqueue work, exit the terminal, and later query status or retrieve results through the /codex:status command.

Architecture Overview

The system splits responsibilities across four specialized modules. State management lives in state.mjs, handling JSON persistence in a temporary directory. Job tracking logic resides in tracked-jobs.mjs, creating log files and updating progress. Enqueueing and process spawning occur in codex-companion.mjs. Finally, status querying and lifecycle enrichment are handled by job-control.mjs.

This separation ensures that the foreground CLI can exit immediately after spawning work, while a detached worker process continues updating shared state that the user can inspect later.

Job State Management

All job metadata is stored as JSON files under the user's temporary folder. The state.mjs module in plugins/codex/scripts/lib/state.mjs provides utilities to load, save, prune, and upsert job records. Each record tracks the job ID, current status, timestamps, process PID, and the path to its dedicated log file.

Because state lives on disk rather than in memory, the system survives parent process termination and allows cross-process status checks.

Enqueueing Background Tasks

When a user invokes a command with the --background (or --wait) flag, the companion script calls enqueueBackgroundTask in plugins/codex/scripts/codex-companion.mjs (lines 84‑99). This function orchestrates the handoff from foreground to background:

1. Create log file: Calls createTrackedProgress to generate a unique log file for the job.
2. Write initial status: Appends "Queued for background execution." to the log.
3. Spawn detached worker: Invokes spawnDetachedTaskWorker to launch a new Node process.
4. Persist record: Stores a "queued" job record containing the child PID and log file path via writeJobFile and upsertJob.

// Enqueue a background task (simplified)
import { createTrackedProgress, appendLogLine, writeJobFile, upsertJob } from "./tracked-jobs.mjs";
import { spawnDetachedTaskWorker } from "./codex-companion.mjs";

function enqueueBackgroundTask(cwd, job) {
  const { logFile } = createTrackedProgress(job);
  appendLogLine(logFile, "Queued for background execution.");

  const child = spawnDetachedTaskWorker(cwd, job.id);
  const queued = { ...job, status: "queued", phase: "queued", pid: child.pid, logFile };
  writeJobFile(job.workspaceRoot, job.id, queued);
  upsertJob(job.workspaceRoot, queued);
  return queued;
}

The Detached Worker Process

The spawnDetachedTaskWorker function (lines 71‑80 in codex-companion.mjs) launches the same script (codex-companion.mjs) with the task-worker subcommand. The child process is spawned with detached: true and stdio: "ignore", allowing the parent to exit while the worker continues.

// Spawn the detached worker (platform‑agnostic)
function spawnDetachedTaskWorker(cwd, jobId) {
  const script = path.join(ROOT_DIR, "scripts", "codex-companion.mjs");
  const child = spawn(process.execPath, [script, "task-worker", "--cwd", cwd, "--job-id", jobId], {
    cwd,
    env: process.env,
    detached: true,
    stdio: "ignore",
    windowsHide: true,
  });
  child.unref();
  return child;
}

The worker receives the job ID, loads the existing state, and executes the actual command logic.

Tracking Progress and Lifecycle

Once running, the worker invokes runTrackedJob from plugins/codex/scripts/lib/tracked-jobs.mjs (lines 42‑104). This function wraps the actual task execution and manages the job lifecycle:

• Progress updates: Writes to the per-job log file via appendLogLine and appendLogBlock.
• Status transitions: Updates the job record from running to completed or failed.
• Result capture: Records timestamps, exit status, and rendered output to the JSON state.

// Detached worker runs the job and records results
import { runTrackedJob } from "./tracked-jobs.mjs";

async function workerMain(job, runner) {
  // `runner` receives a progress reporter bound to the job’s log file
  const execResult = await runTrackedJob(job, () => runner(progress), { logFile: job.logFile });
  // execResult contains { exitStatus, payload, rendered, ... }
}

The log file serves as an append-only audit trail, while the JSON state file provides structured metadata for quick status checks.

Querying Status and Control

The job-control.mjs module in plugins/codex/scripts/lib/job-control.mjs handles user-facing status commands. Its enrichJob function (lines 61‑80) reads the log preview, infers the current phase from the job state, and formats elapsed times for display in /codex:status output.

This module enables users to:

• View live progress without blocking the terminal.
• Identify whether a job is queued, running, or finished.
• Access the full log trail for debugging completed or failed tasks.

Summary

• Persistent state: Job metadata lives in JSON files under the temporary directory, managed by state.mjs.
• Detached execution: enqueueBackgroundTask spawns a detached Node process via spawnDetachedTaskWorker, allowing the parent CLI to exit immediately.
• Log-driven tracking: Each job writes to a dedicated log file using helpers from tracked-jobs.mjs, creating an audit trail of progress.
• Lifecycle management: runTrackedJob handles status transitions, capturing exit codes and timestamps when work completes.
• Status integration: job-control.mjs provides enriched job snapshots for the /codex:status command, enabling asynchronous monitoring.

Frequently Asked Questions

How does the Codex plugin survive after the terminal closes?

The plugin uses spawnDetachedTaskWorker to launch a child process with detached: true and stdio: "ignore". This configuration divorces the worker from the parent's stdio streams and session, allowing it to continue running after the user exits the terminal.

Where is job state stored between commands?

Job state is persisted as JSON files in a temporary directory specific to the user. The state.mjs module handles atomic writes and reads, ensuring that both the foreground CLI and background workers can access consistent job records via writeJobFile and upsertJob.

Can multiple background jobs run simultaneously?

Yes. Each job receives a unique ID, log file, and process PID. The enqueueBackgroundTask function creates independent records for each invocation, and the state directory can hold multiple queued, running, or completed jobs concurrently.

How does the system know when a background job finishes?

The detached worker updates the job record via runTrackedJob, which transitions the status to completed or failed and clears the PID field. When the user runs /codex:status, job-control.mjs reads this persisted state to report the final outcome and elapsed time.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}