How Background Jobs Are Tracked and Managed in the State Module (openai/codex-plugin-cc)

The state module in openai/codex-plugin-cc implements a filesystem-backed job registry that tracks background jobs through an upsert-based API, automatic pruning to 50 entries, and per-job payload/log files isolated by hashed workspace directories.

The state.mjs file serves as the central nervous system for background job persistence in the Codex Companion plugin. Located at plugins/codex/scripts/lib/state.mjs, this module provides thread-safe state management, automatic cleanup, and a clean public API that higher-level components use to orchestrate long-running tasks.

State Storage Layout and Directory Structure

Every workspace receives its own isolated state directory to prevent collisions between concurrent projects. The resolveStateDir function (lines 29-44) builds this path by hashing the workspace root:

// From state.mjs - directory resolution
const hash = crypto
  .createHash("sha256")
  .update(workspaceRoot)
  .digest("hex")
  .slice(0, 16);
const stateDir = path.join(rootDir, hash);

The resulting structure contains:

• state.json – The canonical state file resolved by resolveStateFile (lines 46-48), storing version, configuration, and the jobs array
• jobs/ subdirectory – Individual job payloads (<jobId>.json) and log files (<jobId>.log)

The configurable root directory defaults to the CLAUDE_PLUGIN_DATA environment variable, falling back to a system temporary directory if unset.

Job Representation and the Upsert Pattern

Each job is a plain JavaScript object with mandatory metadata fields. The upsertJob function (lines 29-47) implements an insert-or-update pattern:

Field	Purpose
id	Unique identifier for correlation
createdAt	ISO timestamp set on first insertion
updatedAt	Refreshed on every modification

When a job ID is new, the entry is unshifted (prepended) to the jobs array with fresh timestamps. For existing IDs, the payload is shallow-merged and updatedAt is updated in place.

Automatic Job Pruning and Limits

To prevent unbounded state growth, the module enforces a hard limit via the MAX_JOBS constant (value: 50). The private pruneJobs helper (lines 80-84) executes on every mutation:

// Simplified from state.mjs
function pruneJobs(state) {
  state.jobs.sort((a, b) => 
    new Date(b.updatedAt) - new Date(a.updatedAt)
  );
  state.jobs = state.jobs.slice(0, MAX_JOBS);
}

Jobs sort by updatedAt descending—most recently touched jobs survive, stale jobs are discarded.

Persistent State Writing and File Cleanup

The saveState function (lines 92-115) orchestrates durability and hygiene:

1. Prunes the job list via pruneJobs
2. Removes orphaned job files no longer referenced in the retained list (removeJobFile)
3. Deletes stale log files via removeFileIfExists
4. Ensures directory existence through ensureStateDir
5. Atomically writes JSON to state.json with proper formatting

This guarantees that filesystem state remains consistent even if the process crashes mid-operation.

Per-Job File Operations

For payloads exceeding practical JSON-in-JSON storage or for streaming logs, the module provides granular file helpers:

// From state.mjs around lines 66-71 and 83-91
await writeJobFile(cwd, jobId, { command: "npm test", env: {...} });
const payload = await readJobFile(cwd, jobId);
const logPath = resolveJobLogFile(cwd, jobId);

Helper	Returns	Use Case
writeJobFile(cwd, jobId, data)	Promise<void>	Persist large job payloads
readJobFile(cwd, jobId)	Promise<Object>	Retrieve job-specific data
resolveJobLogFile(cwd, jobId)	string (path)	Stream logs to known location

Public API Reference

The module exports six primary functions consumed by tracked-jobs.mjs and other callers:

// Complete public API from state.mjs
export {
  listJobs,        // (cwd) => Promise<Job[]>
  upsertJob,       // (cwd, jobPatch) => Promise<string>
  setConfig,       // (cwd, key, value) => Promise<void>
  getConfig,       // (cwd) => Promise<Object>
  writeJobFile,    // (cwd, jobId, data) => Promise<void>
  readJobFile,     // (cwd, jobId) => Promise<Object>
  resolveJobLogFile // (cwd, jobId) => string
};

Working Example: Full Job Lifecycle

import {
  upsertJob,
  listJobs,
  writeJobFile,
  resolveJobLogFile,
  setConfig,
} from "./state.mjs";
import fs from "node:fs";

// 1. Register a new background job
const jobId = await upsertJob(process.cwd(), {
  id: "ci-lint-001",
  status: "queued",
  description: "Run ESLint across workspace",
});

// 2. Store execution context separately
await writeJobFile(process.cwd(), jobId, {
  command: "npm run lint",
  workingDirectory: "/home/user/project",
  startedAt: new Date().toISOString(),
});

// 3. Stream logs to dedicated file
const logFile = resolveJobLogFile(process.cwd(), jobId);
fs.appendFileSync(logFile, `[${new Date().toISOString()}] Lint started\n`);

// 4. Update status on completion
await upsertJob(process.cwd(), {
  id: jobId,
  status: "completed",
  result: "success",
  exitCode: 0,
});

// 5. Retrieve current job list
const activeJobs = await listJobs(process.cwd());

// 6. Toggle global configuration
await setConfig(process.cwd(), "stopReviewGate", true);

Integration with Higher-Level Components

The tracked-jobs.mjs module (same directory) wraps this API to expose job commands to the plugin runtime. Meanwhile, tests/state.test.mjs validates critical behaviors: job creation, pruning correctness, and orphaned file cleanup.

Summary

• Isolation: Workspaces hash to unique state directories via resolveStateDir
• Upsert semantics: upsertJob creates or updates with automatic timestamp management
• Hard limits: MAX_JOBS (50) caps growth; pruneJobs enforces recency
• Atomic persistence: saveState prunes, cleans orphaned files, and writes atomically
• Flexible storage: Per-job JSON payloads and log files supplement the main state array
• Minimal API surface: Seven exports cover all job and configuration operations

Frequently Asked Questions

What is the maximum number of jobs the state module retains?

The state module retains 50 jobs maximum, defined by the MAX_JOBS constant in state.mjs. The pruneJobs function sorts by updatedAt and keeps only the most recently touched entries, discarding older jobs during every saveState call.

How does the state module prevent workspace collisions?

It hashes the workspace root path using SHA-256 (truncated to 16 characters) via resolveStateDir to generate unique directory names. This guarantees isolation even when multiple projects run concurrently on the same machine.

What happens to job files when a job is pruned?

During saveState, the module calls removeJobFile for every job ID no longer present in the retained list, then deletes orphaned .log files via removeFileIfExists. This ensures the jobs/ directory stays synchronized with the canonical state.json array.

Can multiple processes safely update state simultaneously?

The current implementation relies on atomic file writes but does not implement file locking. For the Codex Companion plugin's use case (single-node, single-user), this is sufficient. Concurrent modifications from separate processes could theoretically race; the test suite in state.test.mjs covers sequential consistency rather than distributed concurrency.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}