How the Codex Plugin Tracks Background Jobs: File-Based State Management

The Codex plugin implements a lightweight, file-based job tracking system that persists metadata, progress, and output to JSON files in a temporary directory, enabling session-scoped monitoring and automatic pruning of up to 50 jobs.

The openai/codex-plugin-cc repository provides a Codex companion plugin that manages long-running tasks like code reviews and refactoring operations. Understanding how the Codex plugin tracks background jobs is essential for developers extending the tool or debugging asynchronous operations. The system uses a durable, filesystem-backed approach that stores state outside the repository while maintaining session isolation and human-readable logs.

State Storage Architecture

The foundation of the job tracking system resides in a dedicated temporary directory. According to the source code in plugins/codex/scripts/lib/state.mjs, the resolveStateDir function determines the storage location using $TMPDIR/codex-companion/state/, ensuring job data never contaminates the workspace or source control.

Global State and Job-Specific Files

The storage layer maintains two distinct data structures. A global state.json file tracks the aggregate job list, while individual JSON files store per-job metadata. The loadState and saveState functions manage this global registry, while writeJobFile and readJobFile handle atomic updates to specific job records. This separation allows the system to read individual job details without parsing the entire job history.

Job Lifecycle and Execution

Background jobs follow a strict lifecycle managed primarily through plugins/codex/scripts/lib/tracked-jobs.mjs. The runTrackedJob function serves as the central execution wrapper that coordinates state transitions from creation through completion.

Initialization and Record Creation

When a job begins, the system generates a unique identifier via generateJobId and constructs a record using createJobRecord. This record captures essential metadata including createdAt timestamps, optional sessionId values, and filesystem paths. Simultaneously, createJobLogFile initializes a plain-text log file (e.g., <jobId>.log) to capture human-readable progress output.

The following example demonstrates starting a review task with full job tracking:

import { generateJobId, createJobRecord, createJobLogFile, runTrackedJob } from "./tracked-jobs.mjs";

async function startReviewJob(workspaceRoot, reviewTask) {
  const jobId = generateJobId("review");
  const logFile = createJobLogFile(workspaceRoot, jobId, "Codex Review");
  const job = createJobRecord(
    {
      id: jobId,
      workspaceRoot,
      jobClass: "review",
      kind: "review",
      logFile,
    },
    {}
  );

  const runner = async () => {
    // …perform review, return { exitStatus, payload, rendered, threadId, turnId, summary }
  };

  return runTrackedJob(job, runner, { logFile });
}

Running and Completion States

The runTrackedJob implementation writes an initial "running" status containing the process ID (pid) and startedAt timestamp via upsertJob. After the user-supplied runner function resolves, the wrapper updates the job JSON with the final status (completed or failed), completedAt timestamp, result payload, and rendered output. This atomic update ensures the filesystem always reflects the current execution state.

Progress Reporting and Logging

Real-time visibility into long-running tasks relies on the progress reporter pattern. The createProgressReporter function in tracked-jobs.mjs returns a normalized reporting interface that bridges runtime events with persistent storage.

Structured Log Updates

When user code invokes the reporter, the system normalizes the event data and appends a line to the job's log file using appendLogLine or appendLogBlock. Optionally, these updates echo to stderr for immediate terminal visibility. This dual-write approach maintains both a durable audit trail and real-time user feedback without blocking the main execution thread.

To report progress from inside a runner:

import { createProgressReporter } from "./tracked-jobs.mjs";

const reporter = createProgressReporter({ stderr: true, logFile });
reporter({ message: "Searching repository...", phase: "investigating" });

Session Scoping and Status Queries

The plugins/codex/scripts/lib/job-control.mjs module provides higher-level APIs for querying active and historical jobs. The system supports multi-session isolation through the CODEX_COMPANION_SESSION_ID environment variable.

Environment-Based Filtering

The filterJobsForCurrentSession function reads the current session ID via getCurrentSessionId and filters the global job list to return only relevant entries. This ensures that commands like /codex:status display only jobs belonging to the current IDE or terminal session, preventing cross-contamination between different workflow contexts.

Status Snapshot Construction

The buildStatusSnapshot function aggregates running, recent, and latest-finished jobs, then passes each record through enrichJob. This enrichment process calculates elapsed time, infers a human-friendly phase using inferLegacyJobPhase, and previews the last log lines. The resulting snapshot provides the UI layer with concise, contextualized job status without requiring deep filesystem traversal.

Retrieve a status snapshot for the current session:

import { buildStatusSnapshot } from "./job-control.mjs";

const snapshot = buildStatusSnapshot(process.cwd(), { env: process.env });
console.log("Running jobs:", snapshot.running);
console.log("Recent jobs:", snapshot.recent);

Job Cancellation

For cancelling active jobs, the resolveCancelableJob function locates the job record and workspace root, enabling targeted termination:

import { resolveCancelableJob } from "./job-control.mjs";

async function cancelJob(jobId) {
  const { workspaceRoot, job } = resolveCancelableJob(process.cwd(), jobId);
  const { upsertJob } = await import("./state.mjs");
  upsertJob(workspaceRoot, { id: job.id, status: "cancelled", completedAt: new Date().toISOString() });
}

Automatic Cleanup and Resource Management

To prevent unbounded growth of temporary files, the system implements automatic pruning. The saveState function in state.mjs enforces a MAX_JOBS limit of 50 entries, retaining only the newest jobs while removing older JSON files and their associated logs. This hard limit ensures that long-running editor sessions or CI environments do not exhaust disk space with stale job artifacts.

Summary

• File-based persistence: Job data lives in $TMPDIR/codex-companion/state/ outside the repository, using state.json for the global registry and individual JSON files per job.
• Three-module architecture: state.mjs handles persistence, tracked-jobs.mjs manages execution and logging, and job-control.mjs provides querying and session filtering.
• Atomic lifecycle updates: runTrackedJob coordinates status transitions from "running" to "completed" or "failed" with timestamps and process IDs.
• Session isolation: The CODEX_COMPANION_SESSION_ID environment variable enables multi-context workflows through filterJobsForCurrentSession.
• Automatic pruning: The system maintains a hard limit of 50 jobs (MAX_JOBS), automatically cleaning older records and log files to manage disk usage.

Frequently Asked Questions

Where does the Codex plugin store background job data?

The plugin stores all job metadata and logs in a temporary directory located at $TMPDIR/codex-companion/state/, as resolved by the resolveStateDir function in state.mjs. This location stores a global state.json file plus individual JSON files and .log files for each job, ensuring data persists across process restarts while remaining outside the Git repository.

How does the plugin track progress for long-running jobs?

During execution, the createProgressReporter function in tracked-jobs.mjs returns a reporter that normalizes progress events and writes them to the job's log file via appendLogLine. These updates optionally echo to stderr for real-time visibility, while the upsertJob function synchronizes the job's JSON record with the latest metadata and timestamps.

What limits exist on the number of tracked jobs?

The system enforces a hard limit of 50 concurrent and historical jobs through the MAX_JOBS constant. When saveState persists the global registry, it automatically prunes older entries and deletes their associated JSON and log files, preventing unbounded disk usage in long-running environments.

How does session scoping prevent job leakage between workspaces?

The plugin respects the CODEX_COMPANION_SESSION_ID environment variable for multi-session isolation. Functions like filterJobsForCurrentSession in job-control.mjs filter the global job list to include only entries matching the current session ID, ensuring that status commands and cancellation operations affect only jobs initiated from the current context rather than leaking across IDE windows or terminal sessions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}