# Incremental Graph Updating in code-review-graph: Architecture and Performance

> Discover incremental graph updating in code-review-graph. Our architecture re-parses only modified files, updating the knowledge graph in under a second using SHA-256 hashing and parallel processing.

- Repository: [Tirth Kanani/code-review-graph](https://github.com/tirth8205/code-review-graph)
- Tags: architecture
- Published: 2026-07-19

---

**Incremental graph updating re-parses only modified files and their transitive dependents, using SHA-256 content hashing and parallel processing to update the knowledge graph in under a second for typical development cycles.**

The `tirth8205/code-review-graph` project constructs a knowledge graph of code repositories by parsing source files into nodes and edges. For large codebases, full rebuilds are prohibitively expensive, so the system implements **incremental graph updating** to maintain near-real-time accuracy. This approach processes only files that changed since the last update plus any files that depend on them, dramatically reducing computation time while ensuring graph consistency.

## How Incremental Graph Updating Works

### Detecting File Changes via VCS

The process begins in [`code_review_graph/incremental.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/incremental.py) where the `incremental_update` function calls `get_changed_files` to execute a VCS diff. For Git repositories, this runs `git diff --name-only -z` to identify added, modified, or deleted files since the base revision. When the `changed_files` parameter is not explicitly supplied, the system automatically detects changes by comparing against stored VCS metadata.

### Traversing the Dependency Graph

For each changed file, the `find_dependents` function queries the graph's edge tables to locate files that import from or otherwise reference the modified file. The search traverses up to `_MAX_DEPENDENT_HOPS` hops through the dependency graph, returning a `DependentList` that may be truncated if the set exceeds `_MAX_DEPENDENT_FILES`. This ensures semantic changes propagate to all affected downstream code without re-parsing the entire repository.

### Content Hashing and Short-Circuiting

Before parsing, the system filters the work set through two mechanisms. First, `_should_ignore` checks files against repository-wide ignore patterns and validates language detection via `parser.detect_language`. Deleted files are immediately removed from the graph using `store.remove_file_data`. For remaining files, the system computes a SHA-256 hash of current contents and compares it against the hash stored in the `GraphStore`. If the hashes match, the file is skipped entirely, avoiding redundant parsing even if the file appeared in the dependency search.

### Parallel Parsing and Atomic Storage

Files requiring parsing are processed either serially (when the environment variable `CRG_SERIAL_PARSE=1` is set) or in parallel using a `ProcessPoolExecutor` or `ThreadPoolExecutor` selected by `_select_executor_kind`. The `_parse_single_file` function handles Tree-sitter parsing, returning nodes, edges, and the new file hash. With up to `_MAX_PARSE_WORKERS` (default 8) concurrent workers, CPU-bound parsing scales linearly with available cores. All mutations are wrapped in a single SQLite transaction via `store.commit` to maintain consistency.

### Selective Language Resolver Execution

After the primary update, the system conditionally executes language-specific resolvers for ReScript, Spring, Temporal, and HCL only if files of those languages changed. This avoids unnecessary computational work when updates are limited to other languages.

## How Fast Is Incremental Graph Updating?

The performance advantages of incremental graph updating stem from three architectural optimizations:

- **Work-set reduction**: Instead of parsing all tracked files via `collect_all_files`, the incremental path processes only changed files plus their dependents. In typical development cycles, this reduces the work set from thousands of files to a handful.

- **Hash-based deduplication**: The SHA-256 content check eliminates parsing for dependent files whose contents remain unchanged, further shrinking the computational load regardless of dependency graph complexity.

- **Parallel execution**: When the work set exceeds eight files, parsing distributes across up to eight workers, achieving near-linear speedup on multi-core machines where Tree-sitter parsing releases the GIL.

Empirical benchmarks in [`code_review_graph/token_benchmark.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/token_benchmark.py) demonstrate that incremental updates complete in **under one second** for a few dozen changed files on modern 8-core hardware. In contrast, a full rebuild of a 10,000-file repository takes **tens of seconds**, making incremental updating essential for interactive development workflows.

## Running Incremental Updates

You can trigger an incremental update programmatically or via the CLI.

Using Python:

```python
from pathlib import Path
from code_review_graph.incremental import incremental_update
from code_review_graph.graph import GraphStore

repo_root = Path("/path/to/your/repo")
store = GraphStore()                     # Open the SQLite graph store

result = incremental_update(repo_root, store)
print("Updated files:", result["files_updated"])
print("Changed files:", result["changed_files"])
print("Dependent files:", result["dependent_files"])

```

Using the command line:

```bash
$ crg incremental --repo /path/to/repo

```

The CLI command invokes `incremental_update` with the default base of `HEAD~1`, processing changes since the previous commit.

## Summary

- **Incremental graph updating** in `tirth8205/code-review-graph` limits re-parsing to changed files and their transitive dependents, implemented primarily in [`code_review_graph/incremental.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/incremental.py).
- The system uses VCS diffs (`git diff --name-only`) to detect changes and traverses dependency edges via `find_dependents` to locate affected files up to configurable hop and count limits.
- SHA-256 content hashing skips files that haven't changed, while parallel parsing with up to 8 workers accelerates CPU-bound operations.
- Language-specific resolvers run only when needed, and all updates occur within a single SQLite transaction for consistency.
- Typical updates complete in under a second compared to tens of seconds for full repository rebuilds.

## Frequently Asked Questions

### What triggers an incremental graph update in code-review-graph?

An incremental update triggers when you call `incremental_update()` with a repository path, or run the `crg incremental` CLI command. By default, the system compares the current working directory against `HEAD~1` using `get_changed_files`, though you can supply a specific `changed_files` list to bypass VCS detection and manually specify the work set.

### How does code-review-graph handle deleted files during incremental updates?

During the filtering phase in [`incremental.py`](https://github.com/tirth8205/code-review-graph/blob/main/incremental.py), the system checks file existence on disk. Files that no longer exist are removed from the graph using `store.remove_file_data` before the parsing phase begins, ensuring the knowledge graph stays synchronized with the repository state without retaining stale nodes.

### What is the maximum number of dependent files processed in an incremental update?

The `find_dependents` function respects two limits defined in [`incremental.py`](https://github.com/tirth8205/code-review-graph/blob/main/incremental.py): `_MAX_DEPENDENT_HOPS` controls the traversal depth through the dependency graph, while `_MAX_DEPENDENT_FILES` caps the total number of dependents returned. If the dependent set exceeds this threshold, the list is truncated to prevent unbounded growth during updates.

### Can incremental graph updating run in serial mode?

Yes. Setting the environment variable `CRG_SERIAL_PARSE=1` forces the system to use serial parsing instead of parallel execution. This is useful for debugging, very small repositories where process overhead exceeds parallel benefits, or environments with limited resources where process pool creation is undesirable.