# Code Review Graph Benchmark Results for Token Efficiency vs Whole-Corpus Reading

> Discover code review graph benchmark results. Achieve 65x token reduction vs whole-corpus reading, saving significant resources. Explore real-world repository data now.

- Repository: [Tirth Kanani/code-review-graph](https://github.com/tirth8205/code-review-graph)
- Tags: performance
- Published: 2026-08-09

---

**The code-review-graph tool achieves a median 65× token reduction compared to whole-corpus reading, with reductions ranging from 36× to 376× across six real-world repositories.**

The `code-review-graph` open-source project dramatically reduces LLM token consumption by replacing whole-repository context with targeted graph queries. Understanding the benchmark results for token efficiency helps developers evaluate the cost savings of structured code review versus naïvely feeding entire source trees to language models.

## How Token Efficiency Is Measured

The benchmark methodology, documented in [`docs/REPRODUCING.md`](https://github.com/tirth8205/code-review-graph/blob/main/docs/REPRODUCING.md), establishes two distinct measurements:

1. **Whole-corpus baseline**: The raw token count required to feed every source file of a repository to an LLM
2. **Graph-query cost**: The token count of the structured response returned by the graph (search hits plus neighbor edges) for a typical review question

The evaluation runs across six real-world open-source projects totaling 13 commits, comparing the full-corpus baseline against graph-query costs using deterministic embeddings and pinned upstream SHAs.

## Benchmark Results for Token Efficiency

The results demonstrate dramatic reductions in token usage when using the graph-based approach versus whole-corpus reading.

| Repository | Snapshot SHA | Naïve-Corpus Tokens | Avg Graph Tokens | Reduction |
|------------|--------------|--------------------:|-----------------:|----------:|
| fastapi | `22381558` | 948,793 | 2,653 | **375.6×** |
| flask | `a29f88ce` | 143,594 | 2,196 | **71.0×** |
| code-review-graph | `84bde354` | 208,821 | 3,190 | **68.1×** |
| gin | `5c00df8a` | 166,868 | 2,766 | **61.9×** |
| httpx | `b55d4635` | 142,356 | 2,661 | **60.6×** |
| express | `b4ab7d65` | 136,052 | 3,936 | **36.0×** |

*Source: README.md Benchmarks section*

### Key Metrics

According to the benchmark data published in the README, the **median per-question token reduction is approximately 65×**. The efficiency gains range from **36×** (Express) to **376×** (FastAPI), with larger codebases like FastAPI showing the most dramatic savings due to their substantial whole-corpus token counts.

## Technical Architecture Behind Token Savings

The token efficiency stems from the tool's structural extraction approach. Rather than processing raw text, `code-review-graph` uses Tree-sitter to extract functions, classes, calls, and imports into a compact SQLite database.

At query time, the system returns only directly relevant nodes plus a limited "blast-radius" of neighbors (default ≤500 nodes). This typically yields responses of approximately 2,000–3,500 tokens instead of the 100,000-plus tokens required for complete source trees.

The `context_savings` metadata attached to MCP responses records exact token savings for each call, making the benefit transparent to downstream agents.

## Running the Token Efficiency Benchmark

You can reproduce these benchmark results for token efficiency using the provided scripts and modules.

To run the full benchmark suite:

```bash
code-review-graph eval

```

For the standalone token comparison:

```bash
python -m code_review_graph.token_benchmark

```

Programmatically, invoke the benchmark using:

```python
from code_review_graph.eval.benchmarks.token_efficiency import run_token_efficiency

store = ...          # GraphStore from code_review_graph.main.build_graph()

repo = ...           # Repository object for the target

results = run_token_efficiency(store, repo)

print(f"Corpus tokens: {results['naive_corpus_tokens']}")
print(f"Graph tokens : {results['avg_graph_tokens']}")
print(f"Reduction    : {results['avg_graph_tokens'] / results['naive_corpus_tokens']:.2f}×")

```

The formal benchmark implementation in [`code_review_graph/eval/benchmarks/token_efficiency.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/eval/benchmarks/token_efficiency.py) compares the full `get_review_context()` JSON output against changed-file content to generate the reduction ratios.

## Summary

- **code-review-graph** achieves a **median 65× token reduction** compared to whole-corpus reading across six test repositories.
- **Token savings range from 36× to 376×**, with larger projects like FastAPI showing the highest efficiency gains.
- The benchmark methodology measures raw corpus tokens against graph-query response tokens, documented in [`docs/REPRODUCING.md`](https://github.com/tirth8205/code-review-graph/blob/main/docs/REPRODUCING.md).
- Structural extraction via Tree-sitter and limited blast-radius queries (≤500 nodes) enable the compact 2,000–3,500 token responses.
- Reproducibility scripts in [`code_review_graph/token_benchmark.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/token_benchmark.py) and [`code_review_graph/eval/benchmarks/token_efficiency.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/eval/benchmarks/token_efficiency.py) allow independent verification of results.

## Frequently Asked Questions

### What is the median token reduction achieved by code-review-graph?

The benchmark results for token efficiency show a **median reduction of approximately 65×** when comparing graph-based queries to whole-corpus reading. This means that for a typical code review question, the tool uses roughly 1/65th of the tokens required to process the entire repository.

### Which repository showed the highest token efficiency in the benchmarks?

FastAPI demonstrated the highest efficiency with a **375.6× reduction**, reducing 948,793 corpus tokens to just 2,653 graph tokens. This represents the maximum savings in the test suite, though results vary by codebase size and structure.

### How can I reproduce these token efficiency benchmarks?

Run `code-review-graph eval` for the full suite or `python -m code_review_graph.token_benchmark` for standalone testing. The reproducibility documentation in [`docs/REPRODUCING.md`](https://github.com/tirth8205/code-review-graph/blob/main/docs/REPRODUCING.md) specifies pinned SHAs and deterministic embeddings to ensure consistent results across machines.

### What causes the variation in token savings between different repositories?

The variation depends on the ratio between total repository size and the localized context needed for review questions. Larger projects like FastAPI (948K tokens) yield higher absolute savings than smaller ones like Express (136K tokens), though both achieve significant reductions over naïve corpus feeding.