# How Incremental Graph Updating Works in code-review-graph: A 10-Step Pipeline

> Discover how incremental graph updating in code-review-graph rebuilds only changed files and their dependents. Learn the 10-step pipeline and optimize your code analysis.

- Repository: [Tirth Kanani/code-review-graph](https://github.com/tirth8205/code-review-graph)
- Tags: how-to-guide
- Published: 2026-08-07

---

**Incremental graph updating in code-review-graph rebuilds only changed files and their dependents by diffing against the last stored commit, walking import/call edges up to two hops, and re-parsing in parallel while skipping unchanged content via SHA-256 hashing.**

The code-review-graph library constructs a knowledge graph mapping symbols and relationships across a codebase. Re-parsing the entire repository after every change is prohibitively expensive for large projects, so the library implements an **incremental graph updating** mechanism that minimizes CPU and I/O overhead while maintaining accuracy. This process is orchestrated in [`code_review_graph/incremental.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/incremental.py) through a deterministic pipeline that detects changes, expands the affected dependency cone, and updates the graph store transactionally.

## The Incremental Update Pipeline

The `incremental_update()` function in [`code_review_graph/incremental.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/incremental.py) executes a ten-step workflow to synchronize the graph with the working directory. Each step targets a specific concern, from VCS interaction to parallel parsing.

### Step 1: Repository Root Detection

The system first locates the project boundary by walking up the directory tree searching for a `.git` or SVN marker. The `find_repo_root()` and `find_project_root()` functions handle this discovery, though operators can override the result by setting the `CRG_REPO_ROOT` environment variable. This ensures the subsequent `git diff` commands execute within the correct repository context.

### Step 2: Diff Base Resolution

To determine what constitutes a "change," the library invokes `resolve_incremental_base()` to identify the last successfully processed commit. It retrieves the stored `git_head_sha` from the graph metadata; if this SHA still exists in the repository, it serves as the diff base. If the SHA is unreachable (due to a history rewrite or fresh clone), the system falls back to a full rebuild to guarantee consistency.

### Step 3: Changed File Discovery

Using the resolved base, the system executes `git diff --name-status -z <base>` (or the SVN equivalent) via `get_changed_files()`. The `_decode_name_status_paths()` helper parses the null-delimited output, ensuring that both the old and new paths of renamed files are captured for processing. This step produces the initial set of files requiring attention.

### Step 4: Stale File Reconciliation

Before adding new data, the system removes orphaned graph elements. The `_reconcile_stale_files()` function identifies files that have been deleted from the repository or are now excluded by ignore patterns, purging their associated nodes and edges from the store. This prevents phantom relationships from polluting downstream analysis.

### Step 5: Dependent File Resolution

Changes propagate through import and call relationships. The `find_dependents()` function traverses the existing graph's edges to collect files that directly or indirectly depend on the changed set. The walk uses `_single_hop_dependents()` iteratively, bounded by `_MAX_DEPENDENT_HOPS` (defaulting to 2) and capped at `_MAX_DEPENDENT_FILES` (500) to prevent combinatorial explosions in highly interconnected codebases.

### Step 6: Filtering and Hash Verification

Not every file in the changed or dependent set requires parsing. The system filters out ignored and binary files, then performs a SHA-256 hash comparison against stored node hashes. If a file's content hash matches the existing record—even if Git reports it as modified (e.g., due to a timestamp change)—the parser skips it, eliminating redundant work.

### Step 7: Parallel File Parsing

The remaining files are distributed across a worker pool for parsing. The `_make_executor()` function selects between process and thread pools based on the runtime context; it defaults to process-based isolation for CPU-bound parsing but falls back to threads when running under FastMCP stdio transport to avoid deadlocks on Windows. Each worker invokes `_parse_single_file()`, reusing a cached `CodeParser` instance to amortize initialization costs.

### Step 8: Graph Store Updates

Parsed nodes and edges are persisted through `store_file_nodes_edges()`. To avoid foreign-key violations in the SQLite backend, deletions are committed before insertions within the same transaction. This ensures the graph remains internally consistent even if the update process is interrupted.

### Step 9: Metadata Persistence

After a successful update, `_store_vcs_metadata()` records the current `last_updated` timestamp, `last_build_type`, branch name, and commit SHA via `store.set_metadata()`. These values become the diff base for the next incremental run, creating a continuous synchronization chain.

### Step 10: Language-Specific Resolvers

Finally, the system invokes only the resolvers for languages that actually changed. Helper functions like `_run_python_resolver()`, `_run_rescript_resolver()`, or `_run_spring_resolver()` execute language-specific post-processing (such as resolving Spring bean references or Temporal workflow calls) without wasting cycles on unaffected language domains.

## Performance Safeguards and Optimizations

The incremental system incorporates several defensive mechanisms to ensure predictable performance:

- **Bounded Dependency Walking**: The `_MAX_DEPENDENT_HOPS` limit prevents the system from re-parsing the entire graph when a low-level utility file changes, while `_MAX_DEPENDENT_FILES` acts as a hard ceiling on the work queue.

- **Content-Addressed Caching**: SHA-256 hashing ensures that file renames without content changes, or mode changes without content changes, do not trigger parser invocations.

- **Adaptive Concurrency**: The `_select_executor_kind()` logic automatically chooses between `ProcessPoolExecutor` and `ThreadPoolExecutor` based on the transport mechanism, balancing throughput against compatibility constraints.

## Practical Implementation

To perform an incremental update in your own tooling, initialize a `GraphStore` and invoke the update function:

```python
from pathlib import Path
from code_review_graph.graph import GraphStore
from code_review_graph.incremental import incremental_update, full_build

# Initialize a GraphStore (SQLite backed)

store = GraphStore(Path("/tmp/myrepo/graph.db"))

# Full rebuild – usually done once after cloning a repo

full_build(Path("/tmp/myrepo"), store)

# Later, perform an incremental update (auto-detects changed files)

result = incremental_update(Path("/tmp/myrepo"), store)

print("Files re-parsed:", result["files_updated"])
print("New nodes added:", result["total_nodes"])
print("New edges added:", result["total_edges"])

```

You can also force an update against a specific baseline, bypassing automatic diff detection:

```python
incremental_update(
    repo_root=Path("/tmp/myrepo"),
    store=store,
    base="v1.2.3",               # any git ref that resolves to a commit

    changed_files=None,          # let the function compute diffs

    reconcile_stale=True,
)

```

## Summary

- **Change detection** relies on `git diff` against the last stored commit SHA, with automatic fallback to full builds when history is unreachable.
- **Impact analysis** walks import and call edges up to `_MAX_DEPENDENT_HOPS` (default 2) to find affected files, capped at 500 files.
- **Efficiency** is achieved through SHA-256 content hashing that skips unchanged files and parallel parsing using process pools.
- **Consistency** is maintained by reconciling stale files before updates and persisting deletions before insertions in the graph store.
- **Extensibility** allows language-specific resolvers to run only for languages present in the changed set.

## Frequently Asked Questions

### How does code-review-graph handle renamed files during incremental updates?

The `get_changed_files()` function in [`code_review_graph/incremental.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/incremental.py) uses `git diff --name-status -z` and `_decode_name_status_paths()` to capture both the old and new paths of renamed files. Both paths enter the dependency analysis, ensuring that edges pointing to the old filename are removed and edges to the new filename are established.

### What happens if the last stored commit no longer exists in the repository?

When `resolve_incremental_base()` cannot locate the stored `git_head_sha` (for example, after a rebase or force-push), the system abandons the incremental path and triggers a full rebuild. This guarantees that the graph does not retain stale references from a diverged history.

### Can I limit how many dependent files are re-parsed when a core library changes?

Yes. The system respects two constants in [`code_review_graph/incremental.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/incremental.py): `_MAX_DEPENDENT_HOPS` (default 2) limits how many levels of imports or calls to traverse, and `_MAX_DEPENDENT_FILES` (default 500) caps the total number of files added to the parsing queue. These bounds prevent a single change to a utility module from triggering a full repository re-parse.

### Does incremental updating work with version control systems other than Git?

The codebase supports SVN alongside Git. The `get_changed_files()` function detects the repository type and executes the appropriate diff command (or SVN equivalent) to discover changed files. However, the metadata storage and base resolution logic primarily targets Git workflows, with SVN support handling basic diff scenarios.