How the Codex Plugin Manages Background Job Tracking and State Persistence

The Codex plugin implements a lightweight, file-based state management system that stores job metadata as JSON files and execution logs as plain text, enabling crash recovery and monitoring across process restarts.

The openai/codex-plugin-cc repository provides a deterministic mechanism for tracking asynchronous tasks through a sandboxed state directory that isolates job data by workspace. This architecture ensures that long-running operations remain observable even when the parent process restarts, with automatic pruning to prevent unbounded disk usage.

File-Based State Architecture

The plugin eschews external databases in favor of a file-based state management approach that serializes job state directly to disk.

State Directory Structure

All persistence lives under a sandboxed folder resolved by resolveStateDir in plugins/codex/scripts/lib/state.mjs. The system checks for the CLAUDE_PLUGIN_DATA environment variable first, defaulting to /tmp/codex-companion when unset. Within this directory, the plugin maintains:

• A central state.json file containing the global job registry
• A jobs/ subdirectory containing individual JSON records (<jobId>.json) and log files (<jobId>.log)

Job ID Generation

Every background task receives a deterministic identifier through generateJobId(), which produces strings in the format job-<timestamp>-<random>. This function, also located in state.mjs, ensures collision-resistant naming without external coordination.

Job Lifecycle and Persistence

The plugin orchestrates job execution through a structured lifecycle that persists state at every transition.

Creating Job Records

Job metadata is encapsulated in JSON objects created by createJobRecord in plugins/codex/scripts/lib/tracked-jobs.mjs. These records include timestamps, current status, execution phase, process ID (pid), and optional session correlation. The writeJobFile function in state.mjs atomically writes these records to jobs/<jobId>.json.

import { generateJobId, createJobRecord, writeJobFile } from "./state.mjs";

const jobId = generateJobId();
const record = createJobRecord({ 
  title: "Long-running analysis",
  workspaceRoot: "/path/to/project" 
});

await writeJobFile("/path/to/project", jobId, record);

Progress Tracking with Updaters

During execution, createJobProgressUpdater (in tracked-jobs.mjs) normalizes progress events and persists incremental updates. This helper merges changed fields—such as phase, threadId, and turnId—into the existing job record via upsertJob, ensuring the disk state reflects real-time progress without rewriting the entire file system tree.

import { createJobProgressUpdater } from "./tracked-jobs.mjs";

const report = createJobProgressUpdater("/path/to/project", jobId);
report({ phase: "compiling", message: "Compiling source files..." });

Executing Tracked Jobs

The runTrackedJob function in tracked-jobs.mjs provides the primary execution wrapper. It writes an initial "running" state, invokes the user-supplied runner async function, then atomically transitions the record to "completed" or "failed" based on execution results. Upon completion, it appends a "Final output" block to the job's log file via appendLogBlock.

import { runTrackedJob } from "./tracked-jobs.mjs";

await runTrackedJob(
  { id: jobId, workspaceRoot: "/path/to/project", title: "Build task" },
  async () => {
    // Long-running logic here
    return { 
      exitStatus: 0, 
      payload: { success: true }, 
      rendered: "Build completed successfully" 
    };
  },
  { logFile: jobLogPath }
);

State Management and Pruning

To prevent unbounded growth, the plugin implements strict retention policies alongside atomic persistence guarantees.

Atomic State Serialization

Global state operations rely on loadState and saveState in state.mjs. These functions serialize the entire state object—including the job registry—atomically to state.json. This atomic serialization ensures crash recovery: if the process terminates mid-write, the previous state remains intact, preventing corruption.

Automatic Job Pruning

Every call to saveState triggers pruneJobs, which enforces a MAX_JOBS limit of 50 entries. The system retains only the newest jobs based on timestamp, automatically invoking removeJobFile and removeFileIfExists to delete stale JSON records and their associated log files. This pruning occurs transparently during regular state updates, requiring no manual intervention.

Session Correlation and Environment Configuration

The plugin supports multi-session workflows through environment variable injection. When CODEX_COMPANION_SESSION_ID (or a custom configured variable) is present, createJobRecord embeds this value into the job record. This allows UI components in job-control.mjs to group related jobs by session, enabling commands like status to filter by companion context.

Code Examples

Running a Complete Tracked Job

import { generateJobId, createJobRecord, writeJobFile } from "./state.mjs";
import { runTrackedJob, createJobProgressUpdater } from "./tracked-jobs.mjs";

const workspaceRoot = "/path/to/my/project";

// Initialize job descriptor
const job = {
  id: generateJobId(),
  workspaceRoot,
  title: "Dependency installation",
  ...createJobRecord({ title: "Dependency installation" })
};

// Persist initial state
await writeJobFile(workspaceRoot, job.id, job);

// Create progress reporter
const progress = createJobProgressUpdater(workspaceRoot, job.id);

// Execute with tracking
await runTrackedJob(
  job,
  async () => {
    progress({ phase: "installing", message: "Installing npm packages..." });
    
    // ... async work ...
    
    return { 
      exitStatus: 0, 
      payload: { installed: 42 }, 
      rendered: "Installed 42 packages" 
    };
  },
  { logFile: job.logFile }
);

Reading Persisted Job History

import { listJobs } from "./state.mjs";

const recentJobs = listJobs("/path/to/my/project");
console.log(`Found ${recentJobs.length} tracked jobs`);

Updating Progress from a Worker Process

import { createJobProgressUpdater } from "./tracked-jobs.mjs";

const report = createJobProgressUpdater("/path/to/project", "job-1684561234-abcd");

// Rich progress update
report({ 
  phase: "testing", 
  threadId: "thread-123",
  turnId: "turn-456",
  message: "Running unit tests..." 
});

// Simple message (auto-normalized)
report("Tests completed");

Summary

• The openai/codex-plugin-cc plugin stores job state in a sandboxed directory under $CLAUDE_PLUGIN_DATA/state or /tmp/codex-companion, with per-workspace isolation via resolveStateDir.
• Job records live as JSON files in jobs/<jobId>.json, accompanied by plain-text logs in jobs/<jobId>.log.
• Atomic serialization through loadState and saveState guarantees crash recovery by writing complete state snapshots.
• The upsertJob and pruneJobs functions enforce a 50-job retention limit (MAX_JOBS), automatically cleaning stale files.
• createJobProgressUpdater and runTrackedJob in tracked-jobs.mjs provide the high-level API for executing tasks while maintaining persistent observability.

Frequently Asked Questions

How does the Codex plugin store job metadata across process restarts?

The plugin writes job metadata as individual JSON files to a workspace-specific jobs/ directory, with a central state.json file maintaining the registry. Because all state exists on disk under resolveStateDir, any process can reload the complete job history using loadState() after a restart.

What limits the number of persisted jobs in the state directory?

The MAX_JOBS constant set to 50 enforces automatic pruning. Every call to saveState triggers pruneJobs, which identifies the newest 50 jobs by timestamp and deletes older entries along with their log files using removeJobFile.

How does progress tracking update job state without corrupting the log files?

The createJobProgressUpdater function normalizes progress events and calls upsertJob, which merges only changed fields into the JSON record. Log files are appended atomically using appendLogLine and appendLogBlock, ensuring that concurrent updates do not corrupt the plain-text output.

Can jobs be correlated across multiple companion sessions?

Yes. When the CODEX_COMPANION_SESSION_ID environment variable is set, createJobRecord embeds this identifier into the job metadata. This allows UI commands in job-control.mjs to filter and group jobs by session, facilitating multi-window or multi-process workflows.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}