How the Codex Plugin Handles Concurrency When Running Multiple Codex Tasks Simultaneously

The Codex plugin manages concurrent execution by treating each request as an independent tracked job with its own UUID, isolated log files, and persistent JSON records, using atomic file-system operations to ensure parallel jobs never interfere with each other.

The openai/codex-plugin-cc repository implements a robust concurrency model that allows multiple Codex tasks to run simultaneously without resource conflicts. By leveraging file-based state management and isolated job contexts in plugins/codex/scripts/lib/tracked-jobs.mjs, the plugin ensures that each task operates independently while maintaining a coherent global view of all operations.

Independent Job Architecture and UUID Assignment

Every Codex request spawns a unique job identified by a UUID. The createJobRecord function in plugins/codex/scripts/lib/tracked-jobs.mjs initializes a job object containing its ID, timestamps, workspace root, and optional session ID. This isolation ensures that concurrent tasks maintain separate state boundaries from the moment of creation.

Job Progress Tracking

The createJobProgressUpdater utility generates a function that writes incremental updates to the job's persistent storage. Each call to the updater modifies only that specific job's record, preventing cross-contamination between simultaneous operations while providing real-time visibility into task execution.

Atomic State Persistence via File System

Concurrency safety relies on atomic file operations rather than in-memory locks. The plugins/codex/scripts/lib/state.mjs module provides writeJobFile and readJobFile functions that perform atomic writes for individual job records, ensuring durability even when multiple processes write concurrently.

Per-Job File Isolation

Because each job writes to its own dedicated JSON file in the workspace's .codex folder, multiple jobs can update their status simultaneously without race conditions. The file system handles atomicity for single-file writes, ensuring that a job's state remains consistent even when other jobs write concurrently.

Global Job Index Coordination

The upsertJob function maintains a lightweight central index in jobs.json that tracks all job IDs, statuses, phases, and PIDs. Using atomic fs.writeFileSync calls, the plugin updates this global registry without corrupting the index during parallel modifications, providing a coherent snapshot of all running and completed tasks.

Broker-Mediated Execution Flow

The app-server-broker.mjs file implements the orchestration layer that manages concurrent task execution. When a request arrives, the broker spawns a new async function that invokes runTrackedJob. For lifecycle operations such as cancellation or rescuing stuck jobs, the broker utilizes utilities from plugins/codex/scripts/lib/job-control.mjs.

The runTrackedJob Lifecycle

The runTrackedJob function in tracked-jobs.mjs manages the complete job lifecycle through four distinct phases:

1. Initialization – Marks the job as status: "running" and records the process PID in the job record
2. Execution – Runs the user-provided runner function containing the actual Codex API call via makeCodexRunner
3. Completion – On success, updates status to completed and writes rendered output to the log file
4. Error Handling – On failure, catches exceptions, writes status: "failed", and propagates the error while preserving the failure state

This async pattern allows the broker to handle many jobs in parallel, with each execution context remaining fully isolated.

Isolated Logging and Log File Management

Concurrent jobs write to independent log files via createJobLogFile, appendLogLine, and appendLogBlock. Since each job's log path is unique (typically stored in .codex/logs/), simultaneous writes never clash or block each other. The createProgressReporter utility enables writing to both stderr and the dedicated job log simultaneously.

Practical Implementation Examples

The following examples demonstrate how to initiate tracked jobs and monitor concurrent execution:

// Initiating a new Codex job with full tracking
import { createJobRecord, createJobProgressUpdater, runTrackedJob } from
  './plugins/codex/scripts/lib/tracked-jobs.mjs';
import { makeCodexRunner } from './plugins/codex/scripts/lib/codex.mjs';

const job = createJobRecord({
  id: crypto.randomUUID(),
  workspaceRoot: '/my/project',
  title: 'Generate README',
  logFile: null,
});

const progress = createJobProgressUpdater(job.workspaceRoot, job.id);
progress({ message: 'Preparing request…' });

await runTrackedJob(job, async () => {
  const runner = makeCodexRunner({ /* …options… */ });
  return await runner.execute();   // returns { exitStatus, payload, rendered, … }
}, { logFile: '/my/project/.codex/logs/README.log' });

// Reporting progress to both stderr and job log
import { createProgressReporter } from
  './plugins/codex/scripts/lib/tracked-jobs.mjs';

const report = createProgressReporter({ stderr: true, logFile: '/tmp/job.log' });
report({ message: 'Fetching suggestions', phase: 'retrieving' });

// Inspecting the global job index for concurrency monitoring
import fs from 'node:fs';
const indexPath = '/my/project/.codex/jobs.json';
const jobs = JSON.parse(fs.readFileSync(indexPath, 'utf8'));
console.log(jobs.map(j => `${j.id}: ${j.status}`));

Summary

• The Codex plugin achieves concurrency through isolated job records with unique UUIDs stored in tracked-jobs.mjs
• Atomic file-system operations in state.mjs ensure safe parallel writes to per-job JSON files and the global jobs.json index
• The app-server-broker orchestrates parallel execution by spawning async runTrackedJob instances for each request, utilizing job-control.mjs for lifecycle management
• Independent log files prevent I/O conflicts between simultaneous tasks
• The file-based architecture survives process restarts, allowing the broker to resume monitoring jobs marked as running after recovery

Frequently Asked Questions

How does the Codex plugin prevent race conditions between simultaneous jobs?

The plugin eliminates race conditions by giving each job its own JSON state file in the .codex directory and using atomic fs.writeFileSync operations for the global jobs.json index. Since jobs never share mutable state files, concurrent writes cannot corrupt each other's data.

Can the plugin resume jobs after a process crash or restart?

Yes. Because job states persist to disk via writeJobFile and the global index maintains the status field, a restarted broker can re-read jobs.json and identify jobs still marked as running. The file-system-driven design ensures durability independent of process lifetime.

What happens if two jobs try to update the global jobs.json index simultaneously?

The upsertJob function uses atomic file writes to update the central index. While Node.js fs.writeFileSync is not atomic across all platforms, the implementation treats the index as lightweight metadata, and the primary job state lives in isolated per-job files, minimizing the risk of index corruption during concurrent updates.

Is there a limit to how many Codex tasks can run simultaneously?

The practical limit depends on system resources (file descriptors, memory, and CPU) rather than architectural constraints. The broker in app-server-broker.mjs spawns async functions for each request without artificial throttling, though the underlying Codex API and hardware resources ultimately govern throughput.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}