How the Codex Plugin Job File Storage System Works: JSON Metadata and Log Persistence

The Codex plugin uses a lightweight file-system-based job store that persists background tasks as paired JSON metadata and plain-text log files within a per-session state directory, enabling durable execution tracking without external database dependencies.

The openai/codex-plugin-cc repository implements a deterministic persistence layer for managing background operations. This system stores every task, review, and rescue job as discrete files under a workspace-specific state directory, making execution history fully inspectable via standard filesystem tools. Understanding how the Codex plugin job file storage functions enables developers to debug runs, extract structured results, and integrate with external monitoring pipelines.

Storage Architecture and File Layout

The plugin organizes all job data under a configurable state directory (default: <workspace>/state). Within this directory, the jobs/ subdirectory contains two file types for every background operation.

Job Metadata Files (JSON)

Each job receives a unique identifier (e.g., task-<uuid>) and a corresponding JSON file at jobs/<job-id>.json. According to the source code in plugins/codex/scripts/lib/state.mjs, these files contain serialized job objects including:

• id: Unique job identifier
• status: Current state (running, completed, or cancelled)
• jobClass: Operation type (task, review, or rescue)
• logFile: Absolute path to the associated log file
• Timestamps: createdAt, startTime, endTime, or finishedAt
• Optional result data and diagnostic references

Execution Logs (Plain Text)

Complementing each JSON file is a .log file at jobs/<job-id>.log. This plain-text stream captures everything the job outputs: reasoning summaries, assistant messages, review results, and diagnostic traces. The separation of metadata and logs allows efficient status checks without parsing large output files.

Core State Management Functions

The persistence logic lives primarily in plugins/codex/scripts/lib/state.mjs, which exposes deterministic path resolution helpers used throughout the codebase.

Path Resolution Helpers

The module exports two critical functions for file path construction:

• resolveJobFile(workspace, jobId) → Returns the absolute path to the job's JSON metadata file
• resolveJobLogFile(workspace, jobId) → Returns the absolute path to the job's log file

These utilities ensure consistent path generation across the plugin, the CLI, and the test suite. As implemented in tests/state.test.mjs, the test suite validates these paths by writing directly to the resolved locations and verifying filesystem state.

In-Memory State Representation

While jobs reside on disk, the plugin maintains an in-memory state object with a jobs array. Each entry mirrors the JSON structure, enabling rapid status checks without filesystem round-trips. The saveState() function synchronizes this array back to individual JSON files on disk.

Data Persistence Workflow

The Codex plugin follows a strict five-step persistence flow when managing background jobs:

1. Create a job – Generate a unique jobId (e.g., task-${Date.now()} or UUID-based)
2. Initialize metadata – Construct a Job object, push it to state.jobs, and write the JSON file via fs.writeFileSync(jobFile, JSON.stringify(job, null, 2))
3. Stream logs – Append all output to the matching .log file using fs.appendFileSync(logFile, output)
4. Update status – Reflect status changes (running → completed or cancelled) in both the in-memory object and the JSON file
5. Prune history – When the job count exceeds the configured cap, remove oldest entries from both the array and filesystem

Job Pruning and Storage Limits

To prevent unbounded disk growth, the plugin implements automatic pruning within the saveState() function. By default, the system retains approximately 50 jobs (configurable). When this cap is exceeded, saveState() removes the oldest entries from the state.jobs array and deletes both their .json and .log files from the filesystem.

This pruning mechanism, tested in tests/state.test.mjs, ensures that only recent job history remains available for inspection while maintaining predictable storage usage.

Programmatic Access Examples

Creating a Job and Writing Logs

import { resolveJobFile, resolveJobLogFile } from "./state.mjs";
import fs from "fs";

function startTask(workspace, payload) {
  const jobId = `task-${Date.now()}`;
  const jobFile = resolveJobFile(workspace, jobId);
  const logFile = resolveJobLogFile(workspace, jobId);

  const job = {
    id: jobId,
    status: "running",
    jobClass: "task",
    logFile,
    createdAt: new Date().toISOString(),
  };

  // Persist metadata
  fs.writeFileSync(jobFile, JSON.stringify(job, null, 2));

  // Stream execution logs
  fs.appendFileSync(logFile, `🛠️ Starting task ${jobId}\n`);
  
  // ... task execution logic ...
  
  // Update completion status
  job.status = "completed";
  job.finishedAt = new Date().toISOString();
  fs.writeFileSync(jobFile, JSON.stringify(job, null, 2));
  fs.appendFileSync(logFile, `✅ Task ${jobId} completed\n`);
}

Reading the Latest Finished Job

import { readLastFinishedJob } from "./state.mjs";

const latest = readLastFinishedJob(workspace);
console.log("Result:", latest?.result);

Pruning Implementation

function pruneJobs(state, maxJobs = 50) {
  while (state.jobs.length > maxJobs) {
    const oldest = state.jobs.shift();   // Remove first entry
    fs.unlinkSync(resolveJobFile(state.workspace, oldest.id));
    fs.unlinkSync(resolveJobLogFile(state.workspace, oldest.id));
  }
}

Summary

• File-system based storage: All job data resides under <workspace>/state/jobs/ as pairs of JSON metadata and plain-text log files
• Deterministic path resolution: The resolveJobFile() and resolveJobLogFile() functions in plugins/codex/scripts/lib/state.mjs generate consistent file paths
• Structured metadata: JSON files contain job status, class, timestamps, and references to log files
• Automatic pruning: The system maintains a rolling window of approximately 50 jobs, deleting old files to manage storage
• External inspectability: Because storage is pure filesystem-based, external processes and CLI commands (codex status, codex result) can read job history without specialized APIs

Frequently Asked Questions

Where does the Codex plugin store job files?

The plugin stores job files in a per-session state directory, defaulting to <workspace>/state/jobs/. This location contains paired .json metadata files and .log execution files for every background task, review, and rescue operation launched by the plugin.

What data format does the Codex plugin use for job storage?

The system uses two distinct formats: JSON files for structured metadata (including job ID, status, class, and timestamps) and plain-text files for execution logs. This separation allows efficient status queries while preserving full execution output for debugging.

How does the plugin handle storage limits?

The saveState() function automatically prunes old jobs when the total count exceeds approximately 50 entries (configurable). This pruning removes both the stale JSON metadata and corresponding log files from the filesystem, ensuring storage usage remains bounded while retaining recent job history.

Can external tools read Codex plugin job files?

Yes. Because the Codex plugin job file storage system uses standard JSON and plain-text files on the local filesystem, any external process, shell script, or monitoring tool can inspect job status by reading the files directly from the state directory without requiring database connections or proprietary APIs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}