How the Codex Plugin Manages Concurrent Codex Tasks: File‑System Isolation and Async Job Tracking

The Codex plugin manages concurrent Codex tasks by treating each request as an independent tracked job with isolated files, atomic writes, and a lightweight global index, enabling safe parallel execution through a file‑system‑driven concurrency model.

The openai/codex-plugin-cc repository implements a robust concurrency model for handling multiple simultaneous Codex API requests. Instead of managing shared memory or complex threading, the plugin leverages file‑system isolation and atomic writes to ensure each concurrent Codex task operates independently without resource contention.

The Tracked Job Architecture

Each Codex request becomes an independent tracked job with its own UUID, log file, and persistent JSON record stored in the workspace’s .codex folder. This isolation is the foundation of how the plugin handles concurrent Codex tasks.

Job Creation and UUID Assignment

The createJobRecord function in plugins/codex/scripts/lib/tracked-jobs.mjs generates a unique job object containing an ID, timestamps, and optional session ID from the environment. This function works alongside createJobProgressUpdater to establish the initial state before any Codex API calls begin.

State Persistence with Atomic Writes

State updates are handled by writeJobFile and readJobFile in plugins/codex/scripts/lib/state.mjs. Because the file system provides atomic writes for single files, multiple jobs can update their own persistent records simultaneously without race conditions. Each job maintains its own JSON file, ensuring that concurrent writes never corrupt shared state.

Global Job Index Management

The upsertJob function maintains a lightweight central index at jobs.json in the .codex directory. This index stores minimal metadata—ID, status, phase, and PID—allowing the broker to maintain a coherent view of all concurrent Codex tasks. The index updates use atomic fs.writeFileSync calls to prevent corruption during high‑concurrency scenarios.

The Broker and Async Execution Flow

The app‑server‑broker (plugins/codex/scripts/app-server-broker.mjs) coordinates parallel execution by spinning new async functions for each incoming request.

When a request arrives, the broker calls runTrackedJob with the following lifecycle:

1. Marks the job as status: "running", records the process PID, and writes the initial record.
2. Executes the user‑provided runner function containing the actual Codex API call.
3. On success, updates the job to status: "completed" and writes rendered output to the job‑specific log.
4. On failure, catches the error, writes status: "failed", and propagates the exception.

This async pattern allows the broker to manage many concurrent Codex tasks without blocking, while the file‑system backing ensures durability across process restarts.

Isolated File System Strategy

The plugin achieves safe parallelism through file isolation. Each concurrent Codex task receives:

• Independent log files: Created via createJobLogFile, appendLogLine, and appendLogBlock in tracked-jobs.mjs
• Dedicated state files: Per‑job JSON records that eliminate write conflicts
• Atomic index updates: The global jobs.json index uses synchronous atomic writes to maintain consistency

Because no two jobs share write targets, the system avoids locking mechanisms entirely. This design also enables crash recovery—a restarted broker can re‑read the index and resume monitoring any jobs still marked as running.

Practical Code Examples

Start a new tracked Codex job:

import { createJobRecord, createJobProgressUpdater, runTrackedJob } from
  './plugins/codex/scripts/lib/tracked-jobs.mjs';
import { makeCodexRunner } from './plugins/codex/scripts/lib/codex.mjs';

const job = createJobRecord({
  id: crypto.randomUUID(),
  workspaceRoot: '/my/project',
  title: 'Generate README',
  logFile: null,
});

const progress = createJobProgressUpdater(job.workspaceRoot, job.id);
progress({ message: 'Preparing request…' });

await runTrackedJob(job, async () => {
  const runner = makeCodexRunner({ /* …options… */ });
  return await runner.execute();   // returns { exitStatus, payload, rendered, … }
}, { logFile: '/my/project/.codex/logs/README.log' });

Report progress to stderr and log files:

import { createProgressReporter } from
  './plugins/codex/scripts/lib/tracked-jobs.mjs';

const report = createProgressReporter({ stderr: true, logFile: '/tmp/job.log' });
report({ message: 'Fetching suggestions', phase: 'retrieving' });

Inspect the global job index:

import fs from 'node:fs';
const indexPath = '/my/project/.codex/jobs.json';
const jobs = JSON.parse(fs.readFileSync(indexPath, 'utf8'));
console.log(jobs.map(j => `${j.id}: ${j.status}`));

Summary

• The plugin treats each Codex request as an independent tracked job with unique UUIDs and isolated files in the workspace’s .codex folder.
• Atomic file writes in state.mjs and per‑job JSON records prevent race conditions between concurrent Codex tasks.
• The app-server-broker.mjs spawns async workers via runTrackedJob, enabling parallel execution without shared memory.
• A lightweight global index (jobs.json) tracks job status and PID using atomic updates.
• The file‑system approach provides crash recovery and survives broker restarts.

Frequently Asked Questions

How does the plugin prevent race conditions between concurrent jobs?

The plugin prevents race conditions by giving each concurrent Codex task its own isolated files for state and logs. Updates to individual job files occur independently, while the global jobs.json index uses atomic fs.writeFileSync calls. Because no two jobs write to the same file simultaneously, the system avoids locking mechanisms entirely.

What happens to running jobs if the broker process restarts?

Because job state persists to the file system, a restarted broker can re‑read the jobs.json index and resume monitoring any jobs still marked as running. The openai/codex-plugin-cc implementation specifically stores job metadata in the .codex directory, allowing for seamless recovery without losing track of in‑progress concurrent Codex tasks.

How does the global job index stay synchronized?

The upsertJob function maintains consistency by writing to the central jobs.json file using atomic operations. Each entry contains only lightweight metadata—ID, status, phase, and PID—keeping write operations fast and minimizing the window for corruption during concurrent updates from multiple tracked jobs.

Can jobs be cancelled or rescued during execution?

Yes, the plugins/codex/scripts/lib/job-control.mjs module provides utilities for cancelling or rescuing jobs. These helpers work with the broker in app-server-broker.mjs to manage the lifecycle of concurrent Codex tasks, allowing operators to intervene when specific jobs hang or fail unexpectedly.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}