# How Token Savings Is Calculated and Verified in code-review-graph

> Learn how code-review-graph calculates token savings by comparing file sets and optionally verifies with OpenAI tiktoken. Optimize your token usage now.

- Repository: [Tirth Kanani/code-review-graph](https://github.com/tirth8205/code-review-graph)
- Tags: how-to-guide
- Published: 2026-07-19

---

**The `code-review-graph` CLI estimates token savings by comparing a 4-characters-per-token approximation of the full file set against the compressed graph context, then optionally verifies these numbers using OpenAI's official tiktoken tokenizer when you pass the `--verify` flag.**

Every brief CLI invocation in `code-review-graph` surfaces a *Token Savings* panel that quantifies how much LLM context window is preserved by using the graph abstraction instead of raw file contents. This article breaks down the estimation heuristic implemented in [`code_review_graph/context_savings.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/context_savings.py), the exact arithmetic used to compute savings percentages, and the optional verification workflow that reconciles estimates against the real GPT-4 tokenizer.

## Estimating the Naïve Baseline

The baseline represents the token count an LLM would consume if it read the entire change set or repository without optimization.

### Character-Based Token Approximation

The helper `estimate_tokens` (lines 13‑32 of [`code_review_graph/context_savings.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/context_savings.py)) treats any JSON-serializable value as a string, counts its characters, and divides by the constant `CHARS_PER_TOKEN = 4`. This yields a fast, if approximate, token count:

```python

# From context_savings.py

CHARS_PER_TOKEN = 4

def estimate_tokens(obj) -> int:
    text = json.dumps(obj) if not isinstance(obj, str) else obj
    return len(text) // CHARS_PER_TOKEN

```

### File Size Summation for Bulk Operations

When the baseline is a file list, the function `estimate_file_tokens` (lines 35‑51) sums the file size in bytes for every path provided and applies the same 4‑chars‑per‑token rule. This avoids reading file contents into memory during the initial estimation phase, keeping the CLI responsive.

## Measuring the Graph-Generated Context

The *returned* token count is derived from the JSON response produced by graph operations such as `get_review_context` or `detect_changes`. The same `estimate_tokens` routine is applied to the response object (or raw string) to obtain `returned_tokens`. This measures the actual payload that will be sent to the LLM after the graph has pruned irrelevant nodes.

## The Savings Calculation Formula

The core arithmetic lives in `estimate_context_savings` (lines 54‑82). It computes absolute savings and the percentage reduction relative to the baseline:

```python

# Simplified logic from estimate_context_savings

saved_tokens = max(0, baseline_tokens - returned_tokens)
saved_percent = round(saved_tokens * 100 / baseline_tokens)

```

The `max(0, ...)` guard prevents negative savings in edge cases where the graph context unexpectedly exceeds the baseline. The result is packaged into a dictionary flagged with `'estimated': True` to indicate that these figures rely on the heuristic rather than a real tokenizer.

## Rendering the Token Savings Panel

`format_context_savings_panel` constructs the boxed CLI display shown during `detect-changes --brief` and `update --brief` (documented in [`docs/USAGE.md`](https://github.com/tirth8205/code-review-graph/blob/main/docs/USAGE.md)). It reports:

- **Full context**: The baseline estimate
- **Graph context used**: The returned estimate
- **Saved**: Absolute tokens and percentage
- **Per-category breakdown**: Derived by `_breakdown_from_response` inspecting response keys to show which graph sections consume tokens.

## Verifying Against the Real Tokenizer

The panel defaults to `estimated: true`. For production-grade accuracy, the CLI accepts a `--verify` flag that invokes `verify_with_tiktoken` (lines 51‑99).

### The tiktoken Verification Workflow

When verification is requested, the tool:

1. Imports the **tiktoken** library and loads the `cl100k_base` encoding used by GPT‑4.
2. Reads every changed file’s *contents* (not just sizes) and tokenizes each with `enc.encode(p.read_text(...))`, summing them into `naive_real`.
3. Serializes the graph response to JSON and tokenizes it to yield `graph_real`.
4. Returns a dict containing `verified_baseline`, `verified_returned`, `saved_tokens`, and `saved_percent`.

The CLI then appends a *Verified (tiktoken)* row to the panel (lines 80‑88 of `format_context_savings_panel`), letting you see the gap between the fast 4‑character approximation and the exact tokenizer counts. This workflow is detailed in the repository’s reproducibility guide at [`docs/REPRODUCING.md`](https://github.com/tirth8205/code-review-graph/blob/main/docs/REPRODUCING.md).

### CLI Usage Examples

Show the estimated savings for current changes:

```bash
code-review-graph detect-changes --brief

```

Cross-check the numbers with the real GPT‑4 tokenizer:

```bash
code-review-graph detect-changes --brief --verify

```

### Programmatic Access

You can invoke the same logic directly in Python:

```python
from pathlib import Path
from code_review_graph.context_savings import (
    estimate_context_savings,
    verify_with_tiktoken,
)

# Fast estimation (heuristic)

estimate = estimate_context_savings(
    original_context={"changed_files": ["app.py", "utils.py"]},
    returned_context={"review": "..."},
)

# Accurate verification (tiktoken)

verified = verify_with_tiktoken(
    repo_root=Path.cwd(),
    changed_files=["app.py", "utils.py"],
    response={"review": "..."},
)

print(estimate)   # {'estimated': True, 'saved_tokens': 1234, 'saved_percent': 94}

print(verified)   # {'verified_baseline': 12921, 'verified_returned': 10835, ...}

```

## Summary

- **Estimation heuristic**: The tool assumes **4 characters per token** (`CHARS_PER_TOKEN`) to quickly approximate counts without reading file contents.
- **Baseline calculation**: Summed file sizes or serialized JSON length divided by 4, handled by `estimate_tokens` and `estimate_file_tokens` in [`context_savings.py`](https://github.com/tirth8205/code-review-graph/blob/main/context_savings.py).
- **Savings math**: `max(0, baseline - returned)` for absolute tokens, rounded percentage for the UI.
- **Verification**: The `--verify` flag triggers `verify_with_tiktoken`, which uses the `cl100k_base` encoder to tokenize actual file contents and the JSON response.
- **CLI integration**: The panel appears in `detect-changes --brief` and `update --brief`, with verified rows added only when requested.

## Frequently Asked Questions

### What is the CHARS_PER_TOKEN constant and why is it set to 4?

**`CHARS_PER_TOKEN = 4`** is a rough statistical average used by the `estimate_tokens` function in [`code_review_graph/context_savings.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/context_savings.py) to convert character counts into token counts without invoking a heavy tokenizer. This heuristic provides a fast, "good enough" estimate for the CLI panel, though the real ratio varies by text content.

### How accurate is the 4-character heuristic compared to tiktoken?

According to the calibration data in [`docs/REPRODUCING.md`](https://github.com/tirth8205/code-review-graph/blob/main/docs/REPRODUCING.md), the 4‑character approximation typically falls within **~1 %** of the actual `cl100k_base` token counts for codebases. The `--verify` flag exists precisely to surface the exact delta when precision matters.

### Can I verify token savings without using the CLI?

**Yes.** Import `verify_with_tiktoken` from `code_review_graph.context_savings` and pass it a repository root, list of changed files, and the response dictionary. It returns verified counts using the same `tiktoken` logic as the CLI, allowing you to audit savings in custom scripts or unit tests.

### Which CLI commands support the --verify flag?

The `--verify` flag is accepted by any command that displays the Token Savings panel, specifically **`detect-changes --brief`** and **`update --brief`** (see [`docs/COMMANDS.md`](https://github.com/tirth8205/code-review-graph/blob/main/docs/COMMANDS.md)). When provided, the command executes the full tiktoken verification workflow before rendering the panel.