# Code-Graph-RAG Performance Benchmarks: GraphLoader, Trie, and String Operation Speed Tests

> Explore Code-Graph-RAG performance benchmarks for GraphLoader, Trie, and string operations. Discover component latency on workloads up to 50,000 nodes.

- Repository: [Vitali Avagyan/code-graph-rag](https://github.com/vitali87/code-graph-rag)
- Tags: performance
- Published: 2026-08-14

---

**Code-Graph-RAG includes three production-ready micro-benchmarks that measure core component latency on synthetic workloads up to 50,000 nodes and relationships.**

The `vitali87/code-graph-rag` repository ships with a dedicated `benchmarks/` directory containing performance tests for its most latency-sensitive subsystems. These benchmarks target the **GraphLoader** JSON parser, the **FunctionRegistryTrie** name resolver, and low-level **string operations**—each critical for scaling code analysis across large repositories. Results are stored under `benchmarks/results/` and updated with each benchmark run.

## GraphLoader Benchmark: JSON Parsing and Node Retrieval

The **GraphLoader benchmark** ([`benchmarks/bench_graph_loader.py`](https://github.com/vitali87/code-graph-rag/blob/main/benchmarks/bench_graph_loader.py)) measures end-to-end graph loading performance on three synthetic dataset sizes.

### What It Measures

- `json.loads` deserialization time
- `GraphLoader.load` full graph construction
- `find_node_by_property` single-node lookup
- `get_relationships` edge traversal
- Summary generation overhead

### Median Timings by Scale

| Nodes | Relationships | JSON Size | `json.loads` | `GraphLoader.load` | `find_node_by_property` | `get_relationships` |
|------:|--------------:|----------:|-------------:|-------------------:|------------------------:|--------------------:|
| 1,000 | 2,000 | 298 KB | **1.001 ms** | **2.040 ms** | 0.030 ms | 0.148 ms |
| 5,000 | 10,000 | 1,538 KB | **5.032 ms** | **10.106 ms** | 0.030 ms | 0.150 ms |
| 20,000 | 50,000 | 6,980 KB | **24.136 ms** | **61.008 ms** | 0.030 ms | 0.152 ms |

Key observations from the source implementation in [`codebase_rag/graph_loader.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/graph_loader.py):

- Node lookup and relationship retrieval remain **sub-millisecond** even at 20K nodes
- Graph construction scales roughly linearly with JSON size
- Raw JSON parsing accounts for 39–40% of total load time

Full result tables are available in [`benchmarks/results/bench_graph_loader_20260315_000043.txt`](https://github.com/vitali87/code-graph-rag/blob/main/benchmarks/results/bench_graph_loader_20260315_000043.txt).

## FunctionRegistryTrie Benchmark: Name Resolution Speed

The **FunctionRegistryTrie benchmark** ([`benchmarks/bench_trie.py`](https://github.com/vitali87/code-graph-rag/blob/main/benchmarks/bench_trie.py)) exercises the trie-based function registry used for fast qualified name resolution and dead-code detection.

### Measured Operations

- Insertion of qualified names
- Exact lookup
- `find_ending_with` suffix search
- `find_with_prefix` prefix search
- Bulk delete (25% of entries)

### Representative Median Timings

| Size | Insert (ms) | Lookup (ms) | `find_ending_with` (ms) | `find_with_prefix` (ms) | Delete 25% (ms) |
|-----:|------------:|------------:|------------------------:|------------------------:|----------------:|
| 1,000 | 0.92 | 0.64 | 0.03 | 0.04 | 0.08 |
| 5,000 | 4.9 | 3.1 | 0.27 | 0.32 | 0.65 |
| 10,000 | 9.8 | 6.5 | 0.55 | 0.61 | 1.31 |
| 50,000 | 48.7 | 33.2 | 2.9 | 3.1 | 6.4 |

The implementation in [`codebase_rag/function_registry.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/function_registry.py) achieves **sub-millisecond prefix and suffix searches** at all tested scales—critical for interactive RAG query latency. Results print to stdout at runtime; no static archive file is maintained for this benchmark.

## String Operations Benchmark: Low-Level Text Processing

The **string-operations benchmark** ([`benchmarks/bench_string_ops.py`](https://github.com/vitali87/code-graph-rag/blob/main/benchmarks/bench_string_ops.py)) validates that Python's built-in string handling meets throughput requirements for AST processing pipelines.

### Median Timings by Call Volume

| Operation | 1,000 calls | 5,000 calls | 20,000 calls |
|-----------|------------:|------------:|-------------:|
| `str.split` | **0.079 ms** | **0.380 ms** | **1.588 ms** |
| `str.endswith` | **0.179 ms** | **0.897 ms** | **3.582 ms** |
| `str.startswith` | **0.146 ms** | **0.722 ms** | **2.920 ms** |
| `str.join` | **0.036 ms** | **0.185 ms** | **0.733 ms** |
| `str.replace` | **0.014 ms** | **0.071 ms** | **0.287 ms** |
| `regex split` (compiled) | **0.418 ms** | **2.033 ms** | **8.051 ms** |
| f-string formatting | **0.029 ms** | **0.146 ms** | **0.593 ms** |
| `import_distance` helper | varies | varies | varies |

Notable findings:

- `str.replace` is fastest at ~5 ns per call
- Compiled regex split is **10–50× slower** than `str.split`—prefer simple string methods where possible
- The custom `import_distance` utility (for measuring module path similarity) is also benchmarked

Complete results are archived in [`benchmarks/results/bench_string_ops_20260315_000043.txt`](https://github.com/vitali87/code-graph-rag/blob/main/benchmarks/results/bench_string_ops_20260315_000043.txt).

## Running the Benchmarks Locally

Execute any benchmark directly after installing development dependencies:

```bash

# Install with benchmark extras

pip install "code-graph-rag[benchmark]"

# Run individual benchmarks

python -m benchmarks.bench_graph_loader
python -m benchmarks.bench_trie
python -m benchmarks.bench_string_ops

```

Each script:
- Generates synthetic test data automatically
- Prints formatted tables to stdout
- Writes timestamped result files to `benchmarks/results/` (except [`bench_trie.py`](https://github.com/vitali87/code-graph-rag/blob/main/bench_trie.py))

## Key Source Files and Results

| File | Purpose |
|------|---------|
| [`benchmarks/bench_graph_loader.py`](https://github.com/vitali87/code-graph-rag/blob/main/benchmarks/bench_graph_loader.py) | Synthetic JSON graph generation and `GraphLoader` timing |
| [`benchmarks/bench_trie.py`](https://github.com/vitali87/code-graph-rag/blob/main/benchmarks/bench_trie.py) | `FunctionRegistryTrie` operation benchmarks |
| [`benchmarks/bench_string_ops.py`](https://github.com/vitali87/code-graph-rag/blob/main/benchmarks/bench_string_ops.py) | Python string method performance tests |
| `benchmarks/results/bench_graph_loader_*.txt` | Archived GraphLoader timings |
| `benchmarks/results/bench_string_ops_*.txt` | Archived string operation timings |
| [`codebase_rag/graph_loader.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/graph_loader.py) | `GraphLoader` implementation |
| [`codebase_rag/function_registry.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/function_registry.py) | `FunctionRegistryTrie` implementation |

## Summary

- **Three production benchmarks** cover graph loading, trie operations, and string processing in `vitali87/code-graph-rag`
- **GraphLoader** parses 20,000-node graphs in ~61 ms with sub-millisecond node lookups
- **FunctionRegistryTrie** maintains sub-millisecond prefix/suffix searches at 50,000 entries
- **String operations** confirm Python built-ins are sufficient for high-throughput AST pipelines
- **All benchmarks** are runnable via `python -m` with automatic result archival

## Frequently Asked Questions

### How do I run the code-graph-rag performance benchmarks?

Install the package with benchmark extras using `pip install "code-graph-rag[benchmark]"`, then execute `python -m benchmarks.bench_graph_loader`, `python -m benchmarks.bench_trie`, or `python -m benchmarks.bench_string_ops`. Each script runs independently and outputs timing tables to stdout.

### What is the largest graph size tested in the GraphLoader benchmark?

The GraphLoader benchmark tests synthetic graphs up to **20,000 nodes and 50,000 relationships** (6.98 MB JSON). At this scale, full graph construction takes approximately 61 ms with node lookups remaining under 0.05 ms.

### Why does the FunctionRegistryTrie benchmark lack result files?

The [`bench_trie.py`](https://github.com/vitali87/code-graph-rag/blob/main/bench_trie.py) script prints results directly to stdout at runtime rather than archiving to disk. This design choice reflects the benchmark's use for quick interactive verification during development.

### Are these benchmarks suitable for CI regression testing?

Yes. Each benchmark exits with non-zero status on failure and produces deterministic output suitable for parsing. The archived result files in `benchmarks/results/` can be diffed across commits to detect performance regressions in `GraphLoader` or string operation changes.