# Token Savings Estimate for code-review-graph: How the 65× Reduction Is Calculated and Verified

> Discover the 65x token savings estimate for code-review-graph. Learn how this reduction is calculated and verified across six repositories, showing 36x to 376x fewer tokens.

- Repository: [Tirth Kanani/code-review-graph](https://github.com/tirth8205/code-review-graph)
- Tags: performance
- Published: 2026-08-06

---

**The code-review-graph library achieves a median token savings of approximately 65× by comparing raw source file tokens against graph-derived context tokens, with empirical validation across six benchmark repositories showing a reproducible range of 36× to 376× fewer tokens per query.**

The token savings estimate for code-review-graph represents a critical efficiency metric for teams integrating LLMs into their code review workflows. This open-source tool reduces LLM context window consumption by extracting structured graph representations from source code rather than submitting entire files or diffs. Understanding the magnitude and reliability of these savings—derived from deterministic benchmarks—enables precise cost optimization for AI-powered development pipelines.

## How Token Efficiency Is Calculated

The core metric relies on the `compute_token_efficiency` function defined in [`code_review_graph/eval/scorer.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/eval/scorer.py) (lines 12-40). This utility compares the token count required to transmit raw source code or diffs against the token count needed for the compressed graph representation.

The function accepts `raw_tokens` and `graph_tokens` parameters, then derives two key outputs:

- **ratio**: The proportion of graph tokens to raw tokens (lower is better)
- **reduction_percent**: The percentage decrease in token usage

This calculation ensures that savings are quantified using the same tokenizer employed by the target LLM, maintaining consistency across measurements.

## Empirical Results: The 65× Median Savings

According to the benchmark documentation in [`docs/REPRODUCING.md`](https://github.com/tirth8205/code-review-graph/blob/main/docs/REPRODUCING.md) (lines 63-66), the project reports a **median token-saving factor of ~65×** across six diverse benchmark repositories. This indicates that graph-based queries typically consume only 1/65th of the tokens required for raw code submission.

The observed range spans **36× to 376×** fewer tokens per question, demonstrating variability based on file size, complexity, and the specific graph extraction depth. These figures originate from real-world codebases pinned to specific commits, ensuring the data reflects practical usage scenarios rather than synthetic optimizations.

## Ensuring Accuracy Through Reproducible Methodology

The reliability of the 65× estimate stems from a deterministic benchmarking suite rather than anecdotal observation. The **token-efficiency benchmark** located at [`code_review_graph/eval/benchmarks/token_efficiency.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/eval/benchmarks/token_efficiency.py) executes a fixed query set against pinned repository states using consistent SHA values, random seeds, and tokenizer versions.

This approach guarantees that repeating the benchmark produces identical token counts, confirming that the median 65× figure represents a stable, reproducible outcome rather than statistical noise.

### Automated Validation of Edge Cases

Accuracy is further enforced by the test suite in [`tests/test_eval.py`](https://github.com/tirth8205/code-review-graph/blob/main/tests/test_eval.py) (lines 30-45), which validates the scorer's logic against boundary conditions. For example, the tests verify behavior when `raw_tokens` equals zero—such as with empty diffs—ensuring the `compute_token_efficiency` function returns sensible defaults (`ratio: 0.0`, `reduction_percent: 0.0`) rather than raising errors or returning infinity.

## Measuring Token Savings in Practice

Developers can replicate these measurements using the library's scorer module:

```python
from code_review_graph.eval.scorer import compute_token_efficiency

# Standard case: 10,000 raw tokens vs. 3,000 graph tokens

result = compute_token_efficiency(raw_tokens=10_000, graph_tokens=3_000)
print(result)

# {

#   'raw_tokens': 10000,

#   'graph_tokens': 3000,

#   'ratio': 0.3,

#   'reduction_percent': 70.0

# }

# Edge case: empty diff (zero raw tokens)

result = compute_token_efficiency(raw_tokens=0, graph_tokens=500)
print(result)

# {'raw_tokens': 0, 'graph_tokens': 500, 'ratio': 0.0, 'reduction_percent': 0.0}

```

These examples mirror the validation logic found in [`tests/test_eval.py`](https://github.com/tirth8205/code-review-graph/blob/main/tests/test_eval.py), demonstrating how the library handles both typical reductions and degenerate inputs.

## Summary

- **Primary metric**: The `compute_token_efficiency` function in [`code_review_graph/eval/scorer.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/eval/scorer.py) calculates token reduction ratios by comparing raw source tokens against graph-derived context.
- **Median savings**: Approximately **65×** fewer tokens required when using graph context instead of raw code.
- **Observed range**: Token savings vary from **36× to 376×** depending on codebase complexity and query type.
- **Validation**: Deterministic benchmarks in [`code_review_graph/eval/benchmarks/token_efficiency.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/eval/benchmarks/token_efficiency.py) and automated tests in [`tests/test_eval.py`](https://github.com/tirth8205/code-review-graph/blob/main/tests/test_eval.py) ensure the estimates are reproducible and handle edge cases robustly.

## Frequently Asked Questions

### How is the token savings estimate calculated in code-review-graph?

The estimate is calculated by the `compute_token_efficiency` function in [`code_review_graph/eval/scorer.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/eval/scorer.py), which divides the graph-derived token count by the raw token count to produce a ratio, then derives the percentage reduction. This method uses the same tokenizer as the target LLM to ensure accurate comparisons.

### What is the accuracy range of the 65× token reduction claim?

The 65× figure represents the median across six benchmark repositories, with actual measurements ranging from **36× to 376×** fewer tokens per query according to [`docs/REPRODUCING.md`](https://github.com/tirth8205/code-review-graph/blob/main/docs/REPRODUCING.md). This variance reflects differences in file sizes and graph complexity, but the benchmark methodology using pinned commits and fixed seeds ensures the results are reproducible.

### Does code-review-graph handle edge cases in token counting?

Yes, the test suite in [`tests/test_eval.py`](https://github.com/tirth8205/code-review-graph/blob/main/tests/test_eval.py) specifically validates edge cases such as zero raw tokens (empty diffs). The `compute_token_efficiency` function returns a ratio of 0.0 and reduction of 0.0% for these scenarios, preventing division errors and providing sensible defaults.

### Where can I reproduce the token efficiency benchmarks?

Reproduction instructions are documented in [`docs/REPRODUCING.md`](https://github.com/tirth8205/code-review-graph/blob/main/docs/REPRODUCING.md), which references the benchmark script at [`code_review_graph/eval/benchmarks/token_efficiency.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/eval/benchmarks/token_efficiency.py). Running this script against the six included benchmark repositories with pinned commits will generate the same token savings statistics (65× median, 36-376× range) using deterministic parameters.