# How Does Caveman 2 Measure and Verify Token Savings?

> Discover how Caveman 2 accurately measures and verifies token savings. Learn how it parses session transcripts and applies benchmarked compression ratios for efficient AI.

- Repository: [Julius Brussee/caveman](https://github.com/JuliusBrussee/caveman)
- Tags: how-to-guide
- Published: 2026-07-06

---

**Caveman 2 estimates token savings by parsing Claude session transcripts, attributing output tokens to specific modes using timestamped logs, and applying a benchmarked compression ratio of 0.65 to calculate what would have been generated without the tool.**

The `caveman` CLI provides transparent analytics for Claude Code usage by measuring actual output tokens against a verified baseline. According to the JuliusBrussee/caveman source code, the tool implements a multi-stage pipeline that combines log parsing, mode detection, and historical benchmarking to produce accurate savings estimates.

## Parsing Session Transcripts

The measurement process begins in [`src/hooks/caveman-stats.js`](https://github.com/JuliusBrussee/caveman/blob/main/src/hooks/caveman-stats.js) with the **`parseSession`** function. This utility reads the `*.jsonl` transcript produced by Claude Code and extracts two critical metrics from each message's `usage` object: `output_tokens` and `cache_read_input_tokens`.

The function walks every line of the JSONL file, maintains a running total, and returns a timestamped list of per-message output tokens along with the aggregated counts. This raw data serves as the foundation for all subsequent savings calculations.

## Detecting and Attributing Modes

Once the session data is parsed, Caveman 2 determines which compression ratio applies to each message.

### Reading the Active Flag

The **`readFlag`** function in [`src/hooks/caveman-config.js`](https://github.com/JuliusBrussee/caveman/blob/main/src/hooks/caveman-config.js) inspects the `.caveman-active` flag file to identify the current mode (such as `full`). This file acts as the source of truth for which Caveman configuration was active during the session.

### Mode Attribution Logic

The **`attributeByMode`** function joins the per-message timestamps from the transcript with the mode transition log stored in `.caveman-mode-log.jsonl`. This allows the system to handle mid-session mode switches accurately.

If the transition log is missing, the system falls back to a "flag-mtime" heuristic using the file modification time. In the absence of any historical data, it assumes the entire session used the current flag state. This attribution step ensures that savings calculations apply the correct compression ratio to each segment of the conversation.

## Calculating Estimated Savings

### Applying the Compression Benchmark

Caveman 2 stores benchmarked compression ratios in the `COMPRESSION` constant object:

```javascript
const COMPRESSION = { full: 0.65 };

```

This value indicates that **full mode** reduces output tokens by approximately 65% compared to standard Claude Code usage. This ratio derives from empirical benchmarks recorded in the `benchmarks/results/` directory.

### The Savings Formula

The **`deriveSavings`** function implements the core calculation:

```javascript
const saved = Math.round(tokens / (1 - ratio)) - tokens;

```

Where `ratio` is the compression factor (0.65 for full mode). This formula estimates the hypothetical token count that would have been required without Caveman, then subtracts the actual observed tokens to determine the savings.

### USD Cost Conversion

The **`priceForModel`** function converts saved tokens into estimated dollar values using the `MODEL_OUTPUT_PRICE_PER_M` table, which contains per-model pricing data. This allows the CLI to display both token counts and monetary savings in its reports.

## Verification Through Testing

The accuracy of these measurements relies on a comprehensive test suite located in **[`tests/test_caveman_stats.js`](https://github.com/JuliusBrussee/caveman/blob/main/tests/test_caveman_stats.js)**. This suite runs the actual `caveman-stats` script against synthetic sessions and asserts on exact output strings to verify:

- Correct token summation from JSONL transcripts
- Proper handling of the `full` compression ratio (0.65)
- Fallback behavior when benchmark data is missing
- Per-mode attribution accuracy, including mid-session mode changes
- USD conversion calculations using the price table
- Compression-pair detection algorithms
- Lifetime history recording and aggregation

By testing against synthetic data with known outcomes, the suite validates that the `deriveSavings` calculations and `attributeByMode` logic produce mathematically correct results across edge cases.

## Reporting and Persistence

### Human-Readable Output

The **`formatStats`** function generates the detailed multi-line report shown when running `caveman-stats`, while **`formatShare`** produces a concise one-liner optimized for social sharing. The `--share` flag explicitly invokes `formatShare` to output a single-line summary with emoji and estimated dollar savings.

### Lifetime History Aggregation

Caveman 2 persists session snapshots to `.caveman-history.jsonl`. The **`aggregateHistory`** function later processes this file to support the `--all` and `--since` flags, collapsing multiple sessions into cumulative totals while avoiding duplicate entries.

## Practical Usage Examples

### View Session Statistics

```bash
caveman-stats

```

This command parses the most recent `.jsonl` transcript, applies mode attribution, and prints a detailed breakdown including output tokens, cache-read tokens, estimated savings, and USD value.

### Generate a Shareable Summary

```bash
caveman-stats --share

```

Output:

```

🪨 Saved 650 output tokens (~$0.0098) across 1 turns this session — caveman.sh

```

### View Lifetime Totals

```bash
caveman-stats --all

```

This aggregates all entries in `.caveman-history.jsonl` to display cumulative statistics across multiple sessions.

### Programmatic Access

```javascript
const { parseSession, attributeByMode, deriveSavings } = require('./src/hooks/caveman-stats.js');

const sess = parseSession('/path/to/session.jsonl');
const mode = require('./src/hooks/caveman-config.js').readFlag('/home/user/.claude/.caveman-active');
const modeLog = require('./src/hooks/caveman-stats.js').readModeLog('/home/user/.claude/.caveman-mode-log.jsonl');

const attribution = attributeByMode({
  messages: sess.messages,
  modeLog,
  mode,
  flagMtimeMs: null,
  outputTokens: sess.outputTokens,
});

const { estSavedTokens, estSavedUsd } = deriveSavings({ 
  byMode: attribution.byMode, 
  model: sess.model 
});

console.log(`Estimated saved tokens: ${estSavedTokens}`);
console.log(`Estimated USD saved:   ${estSavedUsd}`);

```

## Summary

- **Token parsing** occurs via `parseSession` in [`src/hooks/caveman-stats.js`](https://github.com/JuliusBrussee/caveman/blob/main/src/hooks/caveman-stats.js), which extracts `output_tokens` and `cache_read_input_tokens` from Claude Code JSONL transcripts.
- **Mode attribution** uses `attributeByMode` to map timestamps to active modes, handling mid-session switches through `.caveman-mode-log.jsonl`.
- **Savings calculation** applies the formula `Math.round(tokens / (1 - 0.65)) - tokens` when full mode is active, based on the benchmarked 0.65 compression ratio.
- **Verification** relies on [`tests/test_caveman_stats.js`](https://github.com/JuliusBrussee/caveman/blob/main/tests/test_caveman_stats.js), which validates summation logic, mode attribution, and USD conversion against synthetic data.
- **Reporting** functions `formatStats` and `formatShare` present results in both detailed and condensed formats, while `aggregateHistory` enables lifetime analytics.

## Frequently Asked Questions

### How does Caveman 2 handle mode switching mid-session?

Caveman 2 tracks mode transitions through the `.caveman-mode-log.jsonl` file. The `attributeByMode` function joins these timestamps with message timestamps from the transcript to apply the correct compression ratio to each segment of the conversation. If the log is missing, it falls back to the flag file's modification time or assumes the current mode for the entire session.

### Where does the 0.65 compression ratio come from?

The 0.65 value for `full` mode (stored in the `COMPRESSION` constant) derives from empirical benchmarks stored in `benchmarks/results/*.json`. This ratio represents the measured reduction in output tokens when using Caveman compared to standard Claude Code usage without context optimization.

### How accurate are the USD cost estimates?

The USD estimates use the `MODEL_OUTPUT_PRICE_PER_M` table in [`src/hooks/caveman-stats.js`](https://github.com/JuliusBrussee/caveman/blob/main/src/hooks/caveman-stats.js) to calculate costs based on current per-model pricing. These values update with the tool's pricing data and reflect actual API rates, though they represent estimates unless matched to your specific billing tier.

### Can I verify the token savings calculations independently?

Yes. The full calculation logic is exposed through the Node.js API in [`src/hooks/caveman-stats.js`](https://github.com/JuliusBrussee/caveman/blob/main/src/hooks/caveman-stats.js). You can import `parseSession`, `attributeByMode`, and `deriveSavings` to run the same calculations against your own transcript files. The test suite in [`tests/test_caveman_stats.js`](https://github.com/JuliusBrussee/caveman/blob/main/tests/test_caveman_stats.js) also serves as a reference for expected behavior with known input values.