How Job Progress Tracking and Logging Works in OpenAI Codex Companion

Codex Companion implements job progress tracking and logging through a file-based JSON store under .codex/jobs/, a global index for quick lookups, and structured progress events that update phase, thread, and turn metadata in real-time.

The openai/codex-plugin-cc repository provides a lightweight framework for managing asynchronous AI operations. Its job progress tracking and logging system persists execution state to disk, enabling UI components to monitor long-running tasks like code reviews and code generation without requiring an in-memory database. The core implementation resides in plugins/codex/scripts/lib/tracked-jobs.mjs, with supporting utilities in state.mjs and job-control.mjs.

File-Based Job State Persistence

The Job Record Structure

Each asynchronous job receives a unique identifier and persists as a JSON file at <workspaceRoot>/.codex/jobs/<jobId>.json. These records track the complete lifecycle from initialization through completion or failure.

When runTrackedJob initializes a new job, it creates a running record containing the job ID, workspace root, start timestamp, process ID, and initial status. This record is written to both the individual job file and a global index via upsertJob to enable UI listing without filesystem globbing.

// From tracked-jobs.mjs lines 42-55
writeJobFile(job.workspaceRoot, job.id, runningRecord);
upsertJob(job.workspaceRoot, runningRecord);

State Management Utilities

Low-level persistence operations live in plugins/codex/scripts/lib/state.mjs (not shown in the extracted source). This module provides readJobFile, writeJobFile, and upsertJob functions that handle atomic writes and JSON serialization. The tracked-jobs.mjs module consumes these primitives to abstract file I/O away from business logic.

Real-Time Progress Updates

The Progress Updater Pattern

The createJobProgressUpdater(workspaceRoot, jobId) function returns a callback that normalizes incoming events and conditionally updates the persisted record. This optimization ensures the system does not rewrite the entire JSON file on every minor log line, only when meaningful state changes occur.

// From tracked-jobs.mjs lines 70-102
if (normalized.phase && normalized.phase !== lastPhase) {
  lastPhase = normalized.phase;
  patch.phase = normalized.phase;
  changed = true;
}
...
if (changed) {
  upsertJob(workspaceRoot, patch);
  writeJobFile(jobPath, { ...existing, ...patch });
}

Phase, Thread, and Turn Tracking

The progress updater tracks three distinct dimensions of execution state:

• Phase: High-level stages like "fetching", "analyzing", or "completed"
• Thread ID: Conversation thread identifiers for multi-turn interactions
• Turn ID: Individual turns within a thread

When any of these values change, the updater patches the job record and rewrites the file, ensuring the UI always reflects the current execution context.

Structured Logging Implementation

Log File Generation

Alongside the JSON state files, the system creates human-readable plain-text logs via createJobLogFile. These logs capture timestamped events and structured blocks for final output. The appendLogLine utility writes single-line messages, while appendLogBlock writes titled sections for significant events like error reports or completion summaries.

The Progress Reporter Interface

The createProgressReporter factory function returns a configured reporting callback that multiplexes output to multiple destinations simultaneously. According to the source in tracked-jobs.mjs lines 17-32, the reporter:

1. Writes to stderr when enabled (prefixing lines with [codex])
2. Appends timestamped lines to the designated log file
3. Writes titled log blocks for structured data
4. Invokes optional user-provided callbacks for custom handling

export function createProgressReporter({ stderr = false, logFile = null, onEvent = null } = {}) {
  return (eventOrMessage) => {
    const event = normalizeProgressEvent(eventOrMessage);
    if (stderr && event.message) process.stderr.write(`[codex] ${event.message}\n`);
    appendLogLine(logFile, event.message);
    appendLogBlock(logFile, event.logTitle, event.logBody);
    onEvent?.(event);
  };
}

Orchestrating Tracked Jobs

The runTrackedJob Wrapper

The runTrackedJob(job, runner, options) function serves as the primary entry point for executing work with automatic tracking. It accepts a job descriptor, an async runner function containing the actual work, and options for logging and event callbacks.

As implemented in tracked-jobs.mjs lines 42-81, the wrapper:

1. Persists an initial running status
2. Executes the async runner
3. On success, updates status to completed, records final phase, and writes the rendered output to a "Final output" log block
4. Updates the global index via upsertJob

Error Handling and Finalization

When the runner throws, runTrackedJob catches the exception, persists a failed status with the error message, updates the index, and re-throws the error to maintain stack traces. This ensures failed jobs leave a complete audit trail while preserving error propagation for upstream handlers.

try {
  const execution = await runner();
  const completedRecord = {
    status: "completed",
    phase: lastPhase,
    completedAt: Date.now(),
    exitStatus: execution.exitStatus,
    summary: execution.summary,
    payload: execution.payload
  };
  writeJobFile(job.workspaceRoot, job.id, completedRecord);
  upsertJob(job.workspaceRoot, completedRecord);
  appendLogBlock(logFile, "Final output", execution.rendered);
  return execution;
} catch (error) {
  const failedRecord = {
    status: "failed",
    errorMessage: error.message,
    failedAt: Date.now()
  };
  writeJobFile(job.workspaceRoot, job.id, failedRecord);
  upsertJob(job.workspaceRoot, failedRecord);
  throw error;
}

Complete Implementation Example

The following example demonstrates the typical workflow for creating a tracked job, configuring logging, and executing work with progress updates:

import { 
  createJobRecord, 
  createJobProgressUpdater, 
  createProgressReporter, 
  runTrackedJob 
} from "./plugins/codex/scripts/lib/tracked-jobs.mjs";
import { resolveWorkspaceRoot } from "./plugins/codex/scripts/lib/workspace.mjs";

// Initialize job metadata
const job = createJobRecord({
  id: "review-pr-42",
  workspaceRoot: resolveWorkspaceRoot(),
  title: "Review Pull Request #42"
});

// Configure logging to file and stderr
const logFile = `/tmp/${job.id}.log`;
const progress = createProgressReporter({ 
  logFile, 
  stderr: true,
  onEvent: (event) => console.log(`Phase: ${event.phase}`)
});

// Create progress updater for the job index
const updateJob = createJobProgressUpdater(job.workspaceRoot, job.id);

// Execute work with automatic tracking
await runTrackedJob(job, async () => {
  // Update phase and message
  updateJob({ phase: "fetching", message: "Cloning repository..." });
  await cloneRepository();
  
  // Transition to analysis phase with thread tracking
  updateJob({ 
    phase: "analyzing", 
    threadId: "thread-1",
    message: "Running static analysis..." 
  });
  const results = await runAnalysis();
  
  return {
    exitStatus: 0,
    rendered: "## Review Results\n\n✅ All checks passed",

    summary: "No issues found",
    payload: results
  };
}, { logFile, onEvent: progress });

Summary

• Job persistence: Each job stores state as JSON in .codex/jobs/<jobId>.json, updated via writeJobFile and upsertJob from plugins/codex/scripts/lib/state.mjs.
• Progress tracking: The createJobProgressUpdater function in tracked-jobs.mjs optimizes writes by only persisting when phase, threadId, or turnId changes.
• Structured logging: createProgressReporter multiplexes output to stderr, plain-text log files, and user callbacks, supporting both line-by-line and block-based logging.
• Lifecycle management: runTrackedJob handles the complete lifecycle, transitioning jobs from running to completed or failed, and ensures log files contain final outputs and error details.
• Integration: The system integrates with process.mjs for external command execution and render.mjs for formatting results, providing a complete tracking framework for asynchronous AI operations.

Frequently Asked Questions

How does Codex Companion store job state without a database?

Codex Companion uses a file-based job store where each job persists as a JSON file under the workspace's .codex/jobs/ directory. The upsertJob function in plugins/codex/scripts/lib/state.mjs maintains a global index JSON file that maps job IDs to their metadata, enabling fast lookups without scanning the entire directory. This approach requires no external dependencies and works across process restarts.

What is the difference between createJobProgressUpdater and createProgressReporter?

createJobProgressUpdater returns a function that updates the persisted job record (JSON file) when phase, thread, or turn identifiers change, ensuring the UI sees structural state changes. createProgressReporter returns a function that handles logging output to stderr, appends to log files, and triggers callbacks, but does not modify the job's persisted state. Typically, you use both: the updater for state mutations and the reporter for logging side effects.

How does the system handle concurrent updates to the same job file?

The source code delegates actual file operations to plugins/codex/scripts/lib/state.mjs. While the extracted analysis does not show implementation details, the architecture implies atomic writes through writeJobFile operations. The upsertJob function likely performs read-modify-write cycles on the index file, with the JSON-per-job pattern minimizing write collisions since each job has its own file.

Where are the log files physically located, and what format do they use?

Log files are created alongside the job JSON files or at paths specified by the caller via createJobLogFile. They use plain-text format with timestamped lines written by appendLogLine and titled blocks written by appendLogBlock. The createProgressReporter function can write to stderr simultaneously, but the persistent logs are human-readable text files suitable for debugging and audit trails.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}