# Token Savings Estimation Methodology in Code-Review-Graph: Accuracy and Implementation

> Discover code-review-graph's token savings estimation methodology. Learn how it calculates savings by serializing outputs to JSON and comparing token counts. Improve your LLM efficiency.

- Repository: [Tirth Kanani/code-review-graph](https://github.com/tirth8205/code-review-graph)
- Tags: deep-dive
- Published: 2026-07-29

---

**Code-review-graph estimates token savings by serializing workflow outputs to JSON, counting characters, dividing by four to approximate LLM tokens, and comparing the raw token count against the graph-optimized count to calculate a percentage savings ratio.**

The token savings estimation methodology in the `tirth8205/code-review-graph` repository provides developers with a real-time preview of how much LLM context window is conserved by using graph-based change detection instead of sending entire file contents. This estimation powers the "Real-time token savings" metrics shown in the CLI and helps teams gauge the efficiency of their code review workflows before incurring actual API costs.

## How Token Savings Are Calculated

The estimation pipeline operates in three distinct phases: rough token approximation, workflow-level aggregation, and comparative efficiency analysis.

### Character-to-Token Heuristic

At the core of the methodology lies the **`estimate_tokens`** function, defined in [`code_review_graph/eval/token_benchmark.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/eval/token_benchmark.py). This utility accepts any JSON-serializable object, converts it to a string, measures the character length, and applies a standard heuristic of dividing by four.

```python
from code_review_graph.eval.token_benchmark import estimate_tokens

# Example payload from a change detection workflow

payload = {"file": "src/utils.py", "diff": "def helper(): pass"}
rough_tokens = estimate_tokens(payload)  # → ~12 tokens

```

This 1:4 ratio assumes a rough average for English text and code, aligning with common rule-of-thumb conversions used for OpenAI-style tokenizers.

### Workflow-Level Token Aggregation

The benchmark harnesses—such as `benchmark_review_workflow`, `benchmark_architecture_workflow`, and `benchmark_debug_workflow`—instrument the actual graph operations. They invoke the real tools (`get_minimal_context`, `detect_changes_func`, etc.), capture the resulting payloads, and feed each through `estimate_tokens` to accumulate a total token count for the entire workflow.

This aggregation happens transparently during execution, allowing the system to tally both the **raw tokens** (what a naïve implementation would transmit) and the **graph tokens** (what the optimized pipeline actually consumes).

### Efficiency Computation and Savings Ratio

The final calculation occurs in **`compute_token_efficiency`**, located in [`code_review_graph/eval/scorer.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/eval/scorer.py). This function receives the raw and graph token counts, computes the absolute difference, and expresses the reduction as both a ratio and a percentage.

```python
from code_review_graph.eval.scorer import compute_token_efficiency

result = compute_token_efficiency(raw_tokens=10_000, graph_tokens=3_000)
print(result)

# Output:

# {

#   'raw_tokens': 10000,

#   'graph_tokens': 3000,

#   'savings_ratio': 0.7,

#   'savings_percent': 70.0

# }

```

The savings ratio is calculated as `(raw_tokens - graph_tokens) / raw_tokens`, yielding a float between 0 and 1. The percentage simply multiplies this by 100 for human-readable reporting.

## Implementation Details in the Source Code

The methodology is tightly coupled to two primary evaluation modules. According to the source code in `tirth8205/code-review-graph`, the implementation avoids heavy tokenization libraries to remain lightweight, opting instead for deterministic string operations.

**File:** [`code_review_graph/eval/token_benchmark.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/eval/token_benchmark.py)  
**Purpose:** Implements the `estimate_tokens` helper using JSON serialization and character arithmetic.

**File:** [`code_review_graph/eval/scorer.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/eval/scorer.py)  
**Purpose:** Houses `compute_token_efficiency`, which synthesizes the final metrics dictionary consumed by the CLI and test suite.

The test suite in [`tests/test_eval.py`](https://github.com/tirth8205/code-review-graph/blob/main/tests/test_eval.py) validates these calculations, asserting expected integer outputs for simple strings (e.g., `estimate_tokens("hello")` returns approximately 1) and verifying the correctness of the efficiency dictionary structure.

## Accuracy and Limitations of the Estimate

While the **token savings estimation methodology** is effective for relative comparisons, its accuracy is inherently bounded by the simplicity of the underlying heuristic.

* **Heuristic assumptions:** Dividing character count by four ignores language-specific tokenization patterns, punctuation density, and code-specific subword tokenization used by modern LLMs. Code with many short symbols may incur more tokens than the estimate suggests, while verbose English comments might incur fewer.
* **Empirical validation:** The test suite only verifies that the functions return expected integers for trivial inputs and that the math in `compute_token_efficiency` produces correct ratios. No quantitative error bounds or statistical variance tests are enforced against real tokenizer outputs.
* **Intended use case:** The estimate serves as a **preview mechanism** for the CLI's `--brief` output (e.g., `code-review-graph detect-changes --brief`) and for human-readable reports. It is not designed for precise billing calculations or hard API limits, where actual tokenizer libraries should be used instead.

In practice, the methodology reliably indicates the **order of magnitude** of optimization—distinguishing between 10% and 70% savings—but should not be treated as a precise token budget for cost accounting.

## Summary

- **`estimate_tokens`** in [`token_benchmark.py`](https://github.com/tirth8205/code-review-graph/blob/main/token_benchmark.py) uses a character-count-divided-by-4 heuristic to approximate LLM tokens without heavy dependencies.
- **`compute_token_efficiency`** in [`scorer.py`](https://github.com/tirth8205/code-review-graph/blob/main/scorer.py) compares raw and optimized token tallies to calculate savings ratios and percentages.
- The methodology is validated in [`tests/test_eval.py`](https://github.com/tirth8205/code-review-graph/blob/main/tests/test_eval.py) for structural correctness but carries inherent approximation error due to its simplistic tokenization model.
- The estimate is accurate enough for real-time CLI previews and relative efficiency tracking, but inappropriate for exact billing calculations.

## Frequently Asked Questions

### How does code-review-graph estimate token consumption?

The repository estimates token consumption by serializing workflow outputs to JSON strings, counting the resulting characters, and dividing by four. This occurs in the `estimate_tokens` function within [`code_review_graph/eval/token_benchmark.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/eval/token_benchmark.py), providing a fast, lightweight approximation suitable for real-time reporting.

### Where is the token savings calculation implemented?

The final token savings calculation is implemented in [`code_review_graph/eval/scorer.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/eval/scorer.py) via the `compute_token_efficiency` function. It takes raw and graph token counts as inputs, calculates the difference, and returns a dictionary containing the absolute counts, a savings ratio, and a savings percentage.

### Why divide character count by four?

Dividing by four follows a common rule-of-thumb for English text and code that approximates the average character-to-token ratio observed in OpenAI-style tokenizers. While not exact for all languages or code structures, it provides a reasonable approximation without requiring heavy tokenization libraries or API calls.

### Is the token savings estimate suitable for production billing?

No, the estimate is designed for human-readable previews and relative efficiency comparisons only. Because it ignores language-specific tokenization patterns and code-specific subword segmentation, it may deviate from actual LLM token counts. For precise billing or hard context window limits, use the official tokenizer for your specific model.