# How the Codex Plugin Persists Background Jobs: File-Based State Management

> Learn how the Codex plugin persists background jobs using file based state management. It saves job states as JSON files for immediate durability.

- Repository: [OpenAI/codex-plugin-cc](https://github.com/openai/codex-plugin-cc)
- Tags: internals
- Published: 2026-07-16

---

**The Codex plugin persists background jobs as JSON files in the workspace's `.codex/jobs` directory, using synchronous file operations to guarantee immediate durability of running, completed, and failed job states.**

The openai/codex-plugin-cc repository implements durable background job processing by writing every state change directly to the filesystem. When you initiate a tracked job, the system creates a dedicated JSON file containing metadata and a separate log file for console output, ensuring that job progress survives process crashes or unexpected terminations.

## Job Storage Architecture and File Locations

Background job state is stored in the workspace’s `.codex/jobs` directory. The exact file paths are resolved by helper functions defined in **`plugins/codex/scripts/lib/state.mjs`**.

Each job receives two files:
- A **job file** (JSON) that stores metadata including status, timestamps, IDs, and result payloads
- A **log file** (text) that captures the job’s console output

This separation allows the system to append streaming output without rewriting the entire metadata structure on every line of text.

## Creating and Updating Running Records

### Initializing Job State

When a job starts, the **`runTrackedJob`** function (located in `plugins/codex/scripts/lib/tracked-jobs.mjs`) creates a **running record** containing the initial job metadata. This record is immediately written to disk via two operations:

1. **`writeJobFile`** persists the job-specific JSON file
2. **`upsertJob`** updates the global job index that tracks all jobs in the workspace

The system also checks for the **`CODEX_COMPANION_SESSION_ID`** environment variable (exposed by `SESSION_ID_ENV`) and optionally tags the job with the creating session’s ID for audit trails.

### Real-Time Progress Updates

Long-running jobs report incremental progress through the **`createJobProgressUpdater`** factory. This returns a function that accepts normalized event objects containing fields like `phase`, `threadId`, and `turnId`.

When called, the updater performs a patch operation:
- It reads the existing job file
- Updates only the fields that have changed
- Writes the patched data back to both the job file and the global index

This minimizes disk I/O while keeping the persisted state current.

## Finalizing Job States

### Successful Completion

When a job runner returns successfully, `runTrackedJob` constructs a **completed record**. This JSON object includes:
- Final `exitStatus` (typically `0`)
- The result `payload`
- `rendered` output string
- Completion timestamps

The final output block is appended to the log file via **`appendLogBlock`**, ensuring the complete console history is preserved alongside the metadata.

### Error Handling and Failed Records

If the job runner throws an error, the catch block in `runTrackedJob` reads the existing job file (or falls back to the in-memory running record if the file is unavailable). It then writes a **failed record** containing:
- The error message
- Failure timestamps
- The final state transition

The global job index is updated synchronously to reflect the terminal failure state, ensuring that querying tools (such as those in `plugins/codex/scripts/lib/job-control.mjs`) see the correct status immediately.

## Durability Guarantees Through Synchronous I/O

All file operations use synchronous Node.js APIs: **`fs.writeFileSync`** for JSON metadata and **`fs.appendFileSync`** for log output. This design choice guarantees that the kernel has flushed the data to disk before the function returns, eliminating the window of vulnerability that exists with asynchronous writes where a sudden process crash could lose the last state update.

While synchronous I/O blocks the event loop momentarily, the trade-off is justified for background jobs where state consistency is more critical than microsecond latency.

## Code Examples

### Starting a Tracked Job

```javascript
import { runTrackedJob } from "./lib/tracked-jobs.mjs";

const job = {
  id: "my-job-123",
  workspaceRoot: "/path/to/workspace",
  // optional: title, logFile, etc.
};

async function myRunner() {
  // Do some work...
  return {
    exitStatus: 0,
    payload: { result: "ok" },
    rendered: "Job output",
    threadId: "t1",
    turnId: "turn42",
    summary: "Finished successfully"
  };
}

// Automatically persists running, then completed, records
await runTrackedJob(job, myRunner);

```

### Updating Progress Mid-Execution

```javascript
import { createJobProgressUpdater } from "./lib/tracked-jobs.mjs";

const progress = createJobProgressUpdater("/path/to/workspace", "my-job-123");

// Later in the job logic…
progress({ 
  phase: "processing", 
  threadId: "t1", 
  turnId: "turn5", 
  message: "Halfway there" 
});

```

## Summary

- **File-based persistence**: Job metadata lives in `.codex/jobs` as JSON files, while console output streams to separate log files.
- **Synchronous durability**: The plugin uses `fs.writeFileSync` and `fs.appendFileSync` to ensure state is committed to disk immediately, surviving process crashes.
- **State lifecycle**: Jobs transition through **running**, **completed**, and **failed** records, each written atomically to the job file and global index.
- **Progress tracking**: The `createJobProgressUpdater` function patches only changed fields (phase, threadId, turnId) to minimize I/O while maintaining accuracy.
- **Session correlation**: The `CODEX_COMPANION_SESSION_ID` environment variable optionally tags jobs with their creating session for audit purposes.

## Frequently Asked Questions

### Where does the Codex plugin store background job data?

The plugin stores job data in the workspace’s `.codex/jobs` directory. Each job receives a JSON file for metadata (status, timestamps, IDs) and a separate log file for console output, with paths resolved by `plugins/codex/scripts/lib/state.mjs`.

### What happens if a job fails or the process crashes?

Because the plugin uses synchronous file operations (`fs.writeFileSync`), the last state write is guaranteed to be on disk before the process can crash. If the job throws an error, the catch block writes a **failed record** with the error message and updates the global index, preserving the failure state for later inspection.

### How does the plugin handle real-time progress updates?

Real-time updates are handled by the `createJobProgressUpdater` function in `tracked-jobs.mjs`. It accepts normalized events containing fields like `phase`, `threadId`, and `turnId`, then patches only those specific fields into the existing job file and global index rather than rewriting the entire document.

### Can I identify which session created a specific job?

Yes. The plugin checks for the `CODEX_COMPANION_SESSION_ID` environment variable (defined as `SESSION_ID_ENV`) when creating jobs. If present, this value is stored in the job metadata, allowing you to correlate background jobs with the specific agent or user session that initiated them.