How Token Savings Is Calculated and Verified in code-review-graph

The code-review-graph CLI estimates token savings by comparing a 4-characters-per-token approximation of the full file set against the compressed graph context, then optionally verifies these numbers using OpenAI's official tiktoken tokenizer when you pass the --verify flag.

Every brief CLI invocation in code-review-graph surfaces a Token Savings panel that quantifies how much LLM context window is preserved by using the graph abstraction instead of raw file contents. This article breaks down the estimation heuristic implemented in code_review_graph/context_savings.py, the exact arithmetic used to compute savings percentages, and the optional verification workflow that reconciles estimates against the real GPT-4 tokenizer.

Estimating the Naïve Baseline

The baseline represents the token count an LLM would consume if it read the entire change set or repository without optimization.

Character-Based Token Approximation

The helper estimate_tokens (lines 13‑32 of code_review_graph/context_savings.py) treats any JSON-serializable value as a string, counts its characters, and divides by the constant CHARS_PER_TOKEN = 4. This yields a fast, if approximate, token count:

# From context_savings.py

CHARS_PER_TOKEN = 4

def estimate_tokens(obj) -> int:
    text = json.dumps(obj) if not isinstance(obj, str) else obj
    return len(text) // CHARS_PER_TOKEN

File Size Summation for Bulk Operations

When the baseline is a file list, the function estimate_file_tokens (lines 35‑51) sums the file size in bytes for every path provided and applies the same 4‑chars‑per‑token rule. This avoids reading file contents into memory during the initial estimation phase, keeping the CLI responsive.

Measuring the Graph-Generated Context

The returned token count is derived from the JSON response produced by graph operations such as get_review_context or detect_changes. The same estimate_tokens routine is applied to the response object (or raw string) to obtain returned_tokens. This measures the actual payload that will be sent to the LLM after the graph has pruned irrelevant nodes.

The Savings Calculation Formula

The core arithmetic lives in estimate_context_savings (lines 54‑82). It computes absolute savings and the percentage reduction relative to the baseline:

# Simplified logic from estimate_context_savings

saved_tokens = max(0, baseline_tokens - returned_tokens)
saved_percent = round(saved_tokens * 100 / baseline_tokens)

The max(0, ...) guard prevents negative savings in edge cases where the graph context unexpectedly exceeds the baseline. The result is packaged into a dictionary flagged with 'estimated': True to indicate that these figures rely on the heuristic rather than a real tokenizer.

Rendering the Token Savings Panel

format_context_savings_panel constructs the boxed CLI display shown during detect-changes --brief and update --brief (documented in docs/USAGE.md). It reports:

• Full context: The baseline estimate
• Graph context used: The returned estimate
• Saved: Absolute tokens and percentage
• Per-category breakdown: Derived by _breakdown_from_response inspecting response keys to show which graph sections consume tokens.

Verifying Against the Real Tokenizer

The panel defaults to estimated: true. For production-grade accuracy, the CLI accepts a --verify flag that invokes verify_with_tiktoken (lines 51‑99).

The tiktoken Verification Workflow

When verification is requested, the tool:

1. Imports the tiktoken library and loads the cl100k_base encoding used by GPT‑4.
2. Reads every changed file’s contents (not just sizes) and tokenizes each with enc.encode(p.read_text(...)), summing them into naive_real.
3. Serializes the graph response to JSON and tokenizes it to yield graph_real.
4. Returns a dict containing verified_baseline, verified_returned, saved_tokens, and saved_percent.

The CLI then appends a Verified (tiktoken) row to the panel (lines 80‑88 of format_context_savings_panel), letting you see the gap between the fast 4‑character approximation and the exact tokenizer counts. This workflow is detailed in the repository’s reproducibility guide at docs/REPRODUCING.md.

CLI Usage Examples

Show the estimated savings for current changes:

code-review-graph detect-changes --brief

Cross-check the numbers with the real GPT‑4 tokenizer:

code-review-graph detect-changes --brief --verify

Programmatic Access

You can invoke the same logic directly in Python:

from pathlib import Path
from code_review_graph.context_savings import (
    estimate_context_savings,
    verify_with_tiktoken,
)

# Fast estimation (heuristic)

estimate = estimate_context_savings(
    original_context={"changed_files": ["app.py", "utils.py"]},
    returned_context={"review": "..."},
)

# Accurate verification (tiktoken)

verified = verify_with_tiktoken(
    repo_root=Path.cwd(),
    changed_files=["app.py", "utils.py"],
    response={"review": "..."},
)

print(estimate)   # {'estimated': True, 'saved_tokens': 1234, 'saved_percent': 94}

print(verified)   # {'verified_baseline': 12921, 'verified_returned': 10835, ...}

Summary

• Estimation heuristic: The tool assumes 4 characters per token (CHARS_PER_TOKEN) to quickly approximate counts without reading file contents.
• Baseline calculation: Summed file sizes or serialized JSON length divided by 4, handled by estimate_tokens and estimate_file_tokens in context_savings.py.
• Savings math: max(0, baseline - returned) for absolute tokens, rounded percentage for the UI.
• Verification: The --verify flag triggers verify_with_tiktoken, which uses the cl100k_base encoder to tokenize actual file contents and the JSON response.
• CLI integration: The panel appears in detect-changes --brief and update --brief, with verified rows added only when requested.

Frequently Asked Questions

What is the CHARS_PER_TOKEN constant and why is it set to 4?

CHARS_PER_TOKEN = 4 is a rough statistical average used by the estimate_tokens function in code_review_graph/context_savings.py to convert character counts into token counts without invoking a heavy tokenizer. This heuristic provides a fast, "good enough" estimate for the CLI panel, though the real ratio varies by text content.

How accurate is the 4-character heuristic compared to tiktoken?

According to the calibration data in docs/REPRODUCING.md, the 4‑character approximation typically falls within ~1 % of the actual cl100k_base token counts for codebases. The --verify flag exists precisely to surface the exact delta when precision matters.

Can I verify token savings without using the CLI?

Yes. Import verify_with_tiktoken from code_review_graph.context_savings and pass it a repository root, list of changed files, and the response dictionary. It returns verified counts using the same tiktoken logic as the CLI, allowing you to audit savings in custom scripts or unit tests.

Which CLI commands support the --verify flag?

The --verify flag is accepted by any command that displays the Token Savings panel, specifically detect-changes --brief and update --brief (see docs/COMMANDS.md). When provided, the command executes the full tiktoken verification workflow before rendering the panel.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}