How Headroom Learns from Failed Sessions: Error Tracking and Persistence in TypeScript

Headroom learns from failed sessions by incrementing an in-memory error counter when HTTP requests fail, then persisting these statistics to a per-user JSON-Lines file located at ~/.headroom/session_stats.jsonl for offline pattern analysis.

The chopratejas/headroom repository implements a lightweight observability system that tracks how headroom learns from failed sessions through persistent session statistics. When the TypeScript client encounters compression errors, connection timeouts, or provider-side failures, it captures these events in a structured SessionStats object and flushes them to disk. This mechanism creates a durable record of failure patterns that developers can analyze to identify flaky models or problematic payload configurations.

The SessionStats Data Structure

Headroom defines the statistics contract in src/types/models.ts through the SessionStats interface. This structure maintains granular counters for request outcomes, including a specific failed field that tracks unsuccessful operations.

The client initializes this structure in memory within the HeadroomClient class defined in src/client.ts. The stats() method exposes the current state, allowing real-time inspection of requests.total and requests.failed tallies before they are written to disk.

Detecting and Recording Failures in Real-Time

Request-Level Error Detection

Inside src/client.ts, the private request() method implements a try-catch block that distinguishes successful calls from failures. When the HTTP request succeeds, the client increments this.stats.requests.total. When it catches an exception, it increments this.stats.requests.failed before re-throwing the error.

// Pattern extracted from src/client.ts
private stats: SessionStats = { requests: { total: 0, failed: 0, ... } };

private async request(/* … */) {
  try {
    // …perform the HTTP request…
    this.stats.requests.total += 1;
  } catch (e) {
    this.stats.requests.failed += 1;          // ← count a failure
    throw mapProxyError(e);                   // convert and re-throw
  }
}

Typed Error Conversion

Before throwing, the client converts raw errors into typed Headroom*Error instances via the mapProxyError() utility. This ensures that compression failures, network timeouts, and provider errors are normalized into predictable types while the original failure context is preserved in the stats counter.

Persisting Failure Data to Disk

The session_stats.jsonl File Location

The helper function sessionStatsPath() in src/paths.ts computes the storage location for session data. By default, this resolves to ~/.headroom/session_stats.jsonl, though the path can be overridden via environment configuration. The unit tests in sdk/typescript/test/paths.test.ts verify that this path correctly points to the user's home directory or a specified workspace.

Flushing Stats with flushStats()

When the process exits or when the flushStats() method is called explicitly, the client appends the current SessionStats object to the JSON-Lines file. This append-only approach creates a durable log where each line represents a complete session snapshot.

// Pattern from src/client.ts
private async flushStats() {
  const path = sessionStatsPath();               // → ~/.headroom/session_stats.jsonl
  await fs.appendFile(path, JSON.stringify(this.stats) + "\n");
}

Because the format uses JSON Lines (one JSON object per line), external tools can stream and parse the file without loading the entire history into memory.

Analyzing Failed Sessions for Patterns

Reading Historical Stats

You can consume the persisted logs to compute aggregate metrics across multiple sessions. The line-delimited format allows efficient streaming analysis using Node.js readline interfaces.

import { createReadStream } from "fs";
import { createInterface } from "readline";
import { sessionStatsPath } from "headroom/paths";

async function failureRate() {
  const path = sessionStatsPath();
  const rl = createInterface({ input: createReadStream(path) });

  let total = 0;
  let failed = 0;

  for await (const line of rl) {
    const stats = JSON.parse(line);
    total += stats.requests?.total ?? 0;
    failed += stats.requests?.failed ?? 0;
  }

  console.log(`Failure rate: ${(failed / total) * 100}%`);
}

failureRate();

Calculating Failure Rates by Model

By correlating the failed counter with request metadata (such as model names or payload sizes recorded in your application logs), you can identify specific patterns—such as a particular model that consistently triggers compression errors or a payload size threshold that causes timeouts.

Practical Implementation Example

The following example demonstrates how to trigger a failed request, ensure stats are flushed, and inspect the resulting session data. This pattern is tested in sdk/typescript/test/client.test.ts, which verifies that failed requests increment the stats counter correctly.

import { HeadroomClient } from "headroom";
import { readFileSync } from "fs";
import { sessionStatsPath } from "headroom/paths";

async function demo() {
  const client = new HeadroomClient({ baseUrl: "http://localhost:8787" });

  try {
    // This request is purposefully malformed to cause a compression error.
    await client.compress({
      model: "gpt-4o",
      messages: [{ role: "user", content: "Hello" }],
      // Wrong config: missing required fields, causing the proxy to reject it.
    });
  } catch (err) {
    console.error("Compression failed:", err);
  }

  // Ensure stats are flushed to disk.
  await client.flushStats();

  // Read the most recent entry.
  const statsPath = sessionStatsPath();
  const lastLine = readFileSync(statsPath, "utf-8")
    .trim()
    .split("\n")
    .pop();
    
  console.log("Recorded session stats:", JSON.parse(lastLine));
}

demo();

Summary

• Detection: The request() method in src/client.ts catches errors and increments the failed counter in the SessionStats object.
• Typing: Raw errors are converted to typed Headroom*Error instances via mapProxyError() before being thrown.
• Persistence: The flushStats() method appends statistics to ~/.headroom/session_stats.jsonl, with the path resolved by sessionStatsPath() in src/paths.ts.
• Analysis: The JSON-Lines format supports offline aggregation to calculate failure rates and identify patterns across sessions.
• Foundation: While the repository does not yet implement automatic retraining, the persisted failure log provides the raw signal required for downstream learning pipelines.

Frequently Asked Questions

Where does Headroom store session statistics?

Headroom stores session statistics in a JSON-Lines file located at ~/.headroom/session_stats.jsonl by default. The sessionStatsPath() function in src/paths.ts computes this location, which can be overridden for different workspace configurations. Each line in the file represents a single session's SessionStats object, including the failed request counter.

What types of errors trigger the failed session counter?

The counter increments for any error caught in the request() method within src/client.ts, including compression errors, connection timeouts, and provider-side failures. Before the error is re-thrown, mapProxyError() converts it to a typed error (such as HeadroomCompressError), ensuring consistent error classification while the stats system records the failure regardless of type.

How can I programmatically access failure statistics from previous sessions?

You can read the session_stats.jsonl file directly using Node.js streams or file system APIs. Since the file uses the JSON-Lines format, you can parse it line-by-line to extract historical total and failed counters. The sessionStatsPath() export from headroom/paths provides the correct file location for the current environment.

Is there an automatic retraining mechanism based on failed sessions?

Currently, the repository does not implement an automatic retraining or self-healing pipeline. The system persists failure data to enable offline analysis and debugging, but any learning or model adjustment based on the failed counter must be implemented by external tools that consume the session_stats.jsonl log.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}