How the Codex Plugin Manages and Tracks Background Jobs: File-Based State Architecture Explained

The Codex plugin implements a durable, file-based job tracking system that records every background job's metadata, progress, and output in temporary JSON and log files, enabling persistence across process restarts.

The openai/codex-plugin-cc repository uses a lightweight state management approach that stores runtime data outside the workspace. This design ensures that background operations—such as code reviews or long-running generation tasks—remain tracked even if the companion process restarts, with all state isolated in the OS temporary directory to prevent repository contamination.

File-Based State Architecture

The plugin persists job data in $TMPDIR/codex-companion/state/, creating a clean separation between code and runtime state. This directory contains a global state.json file for aggregate tracking and individual JSON files for each job, alongside plain-text log files capturing human-readable output.

Global State and Job Storage

The state management layer handles persistence through three core operations defined in plugins/codex/scripts/lib/state.mjs:

• resolveStateDir – Determines the platform-specific temporary directory path
• loadState – Reads the global state.json containing the job index
• saveState – Writes aggregated state and enforces the MAX_JOBS limit of 50 entries, pruning older records to prevent unbounded growth

Individual job records store unique IDs, timestamps, status enums ("running", "completed", "failed"), and optional session identifiers. The upsertJob function synchronizes these records between memory and disk, while writeJobFile and readJobFile handle atomic I/O operations for specific job JSON files.

Log Files and Progress Capture

Every job receives a dedicated <jobId>.log file created by createJobLogFile in plugins/codex/scripts/lib/tracked-jobs.mjs. The system appends progress lines via appendLogLine and appendLogBlock, creating a durable audit trail that survives process crashes and enables post-hoc analysis of long-running tasks.

Core Implementation Modules

The job tracking system spans three specialized modules that handle distinct concerns: persistence primitives, execution wrappers, and session-aware querying.

State Management Primitives (state.mjs)

Located at plugins/codex/scripts/lib/state.mjs, this module defines the data layer for the entire system. Key functions include:

• generateJobId – Creates unique identifiers prefixed by job class (e.g., "review-" or "generate-")
• createJobRecord – Initializes metadata objects with createdAt timestamps, workspaceRoot paths, and optional sessionId values
• upsertJob – Atomic update operations that modify both in-memory caches and on-disk JSON files

Job Tracking and Execution (tracked-jobs.mjs)

The plugins/codex/scripts/lib/tracked-jobs.mjs module provides the runtime machinery for job execution:

• runTrackedJob – The primary execution wrapper that writes a "running" status record (including startedAt and pid), invokes the user-supplied runner function, and upon completion updates the job with final status, timestamps, result payloads, and rendered output
• createProgressReporter – Returns a callback function that normalizes progress events, writes formatted lines to the job log, and optionally echoes updates to stderr for real-time visibility

Session-Aware Control Logic (job-control.mjs)

Higher-level UI commands rely on plugins/codex/scripts/lib/job-control.mjs for aggregate operations:

• buildStatusSnapshot – Aggregates running, recent, and latest-finished jobs, enriching each record with inferred phases and elapsed time calculations via enrichJob
• filterJobsForCurrentSession – Isolates jobs belonging to the current companion session by reading the CODEX_COMPANION_SESSION_ID environment variable
• resolveCancelableJob – Validates job IDs and returns resolvable workspace paths for termination operations

Job Lifecycle and Execution Flow

The system follows a strict six-phase lifecycle that ensures data consistency from initiation to archival.

1. Job Creation and Initialization

When initiating a background activity, the plugin calls createJobRecord to initialize a metadata object containing the job ID, workspace root, job class, and log file path. Immediately after, createJobLogFile initializes an empty log file on disk, establishing the persistence layer before any work begins.

2. Running State Capture

The runTrackedJob function writes an atomic "running" entry to the job JSON file, capturing the pid, startedAt timestamp, and initial status. This record serves as a lock file and heartbeat indicator for status queries.

3. Progress Reporting

User code receives a reporter function from createProgressReporter. Calling this function with progress events triggers three operations: normalizing the event data, appending a formatted line to the job’s log file, and updating the job JSON through upsertJob to reflect the latest activity timestamp.

4. Completion and Pruning

Upon runner resolution, runTrackedJob records the final status ("completed" or "failed"), appends the rendered output to the log, and updates the job record with completedAt timestamps and result payloads. The saveState function then enforces the 50-job retention limit, deleting obsolete JSON files and logs to manage storage consumption.

5. Status Querying

UI commands such as /codex:status utilize buildStatusSnapshot to read stored JSON files, compute elapsed times, and generate human-friendly phase descriptions via inferLegacyJobPhase. This operation aggregates across the state directory to present a unified view of system activity.

Session Scoping and Isolation

The plugin supports multi-session isolation through the CODEX_COMPANION_SESSION_ID environment variable. When set, filterJobsForCurrentSession restricts buildStatusSnapshot results to jobs created within that specific session, preventing cross-contamination between different IDE windows or terminal sessions. This scoping ensures that status commands only display relevant background jobs while preserving historical data for potential cross-session auditing.

Practical Implementation Examples

The following patterns demonstrate how to interact with the job tracking system in practice.

Starting a Tracked Background Job

import { generateJobId, createJobRecord, createJobLogFile, runTrackedJob } from "./tracked-jobs.mjs";

async function startReviewJob(workspaceRoot, reviewTask) {
  const jobId = generateJobId("review");
  const logFile = createJobLogFile(workspaceRoot, jobId, "Codex Review");
  
  const job = createJobRecord(
    {
      id: jobId,
      workspaceRoot,
      jobClass: "review",
      kind: "review",
      logFile,
    },
    {}
  );

  const runner = async () => {
    // Perform review work...
    return { 
      exitStatus: 0, 
      payload: reviewResults, 
      rendered: "Review complete", 
      threadId, 
      turnId, 
      summary 
    };
  };

  return runTrackedJob(job, runner, { logFile });
}

Reporting Progress from Within a Job

import { createProgressReporter } from "./tracked-jobs.mjs";

const reporter = createProgressReporter({ stderr: true, logFile });
reporter({ message: "Searching repository...", phase: "investigating" });
reporter({ message: "Analyzing dependencies...", phase: "thinking" });

Querying Session-Scoped Status

import { buildStatusSnapshot } from "./job-control.mjs";

const snapshot = buildStatusSnapshot(process.cwd(), { env: process.env });
console.log("Active jobs:", snapshot.running.length);
console.log("Recent completions:", snapshot.recent);

Canceling an Active Job

import { resolveCancelableJob } from "./job-control.mjs";
import { upsertJob } from "./state.mjs";

async function cancelJob(jobId) {
  const { workspaceRoot, job } = resolveCancelableJob(process.cwd(), jobId);
  // Signal termination via job.pid or update state directly
  upsertJob(workspaceRoot, { 
    id: job.id, 
    status: "cancelled", 
    completedAt: new Date().toISOString() 
  });
}

Summary

• The Codex plugin stores job state in $TMPDIR/codex-companion/state/ as JSON files and plain-text logs, ensuring durability across process restarts.
• Three core modules handle persistence (state.mjs), execution (tracked-jobs.mjs), and querying (job-control.mjs) of background jobs.
• Jobs progress through six lifecycle phases: creation, running state capture, progress reporting, completion, pruning (limited to 50 jobs), and status querying.
• Session isolation via CODEX_COMPANION_SESSION_ID filters job visibility without deleting historical data.
• All state lives outside the repository, preventing git contamination while enabling rich progress tracking and post-hoc log analysis.

Frequently Asked Questions

Where does the Codex plugin store background job data?

The plugin stores all job metadata and logs in the operating system's temporary directory under $TMPDIR/codex-companion/state/. This location contains a global state.json file for indexing and individual JSON files for each job, alongside <jobId>.log files capturing human-readable output. This design ensures that job state persists across companion restarts without polluting the workspace repository.

How does the system prevent unlimited growth of job history?

The saveState function in state.mjs enforces a hard limit of MAX_JOBS (50 entries). When writing state updates, the system retains only the 50 newest job records and automatically deletes older JSON files and associated log files from the temporary directory. This pruning occurs automatically during every state persistence operation.

What is the purpose of session IDs in job tracking?

The CODEX_COMPANION_SESSION_ID environment variable enables multi-session isolation, ensuring that commands like /codex:status only display jobs initiated from the current IDE window or terminal session. The filterJobsForCurrentSession function in job-control.mjs compares this environment variable against each job's sessionId field, filtering the snapshot results accordingly while preserving all historical data for potential cross-session access.

How does runTrackedJob handle job status transitions?

The runTrackedJob wrapper in tracked-jobs.mjs orchestrates atomic status updates: it first writes a "running" record with the process ID and start timestamp, then executes the user-provided runner function, and finally updates the job with either "completed" or "failed" status along with the final payload, rendered output, and completion timestamp. This ensures that every job record contains accurate timing data and terminal state information regardless of how the runner exits.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}