How CRG Handles Incremental Updates for Code Changes: A Technical Deep Dive

Code Review Graph (CRG) handles incremental updates for code changes by detecting modified files since the last build, expanding the scope to include dependent files through graph traversal, re-parsing only the affected subset, and atomically merging the new AST nodes and edges back into its SQLite-backed graph store.

Code Review Graph (CRG) is an open-source Python tool—hosted at tirth8205/code-review-graph—that maintains a queryable graph representation of code relationships. Instead of rebuilding the entire graph after every commit, CRG performs incremental updates for code changes to minimize CPU and I/O overhead. The implementation resides primarily in code_review_graph/incremental.py and orchestrates a nine-step pipeline from Git diff detection to graph persistence.

Step 1: Resolving the Base Commit

The incremental process begins by establishing a diff baseline. The function resolve_incremental_base() (lines 44-66 in code_review_graph/incremental.py) queries the graph metadata for the stored git_head_sha from the previous successful build.

If the stored SHA still points to a reachable commit in the current repository history, CRG uses it as the base for comparison. If the commit is unreachable or the metadata is missing, the function returns None, which triggers a full rebuild via full_build() before any incremental logic proceeds. This ensures the graph never applies partial updates on top of a stale or corrupted state.

Step 2: Detecting Changed Files

Once a base commit is resolved, get_changed_files() (lines 70-87) executes git diff --name-status -z <base> (or SVN equivalents) to enumerate repository changes. The helper _decode_name_status_paths() parses the null-delimited output to return a clean list of paths categorized as added, modified, renamed, or deleted.

This raw change set represents the minimal surface area of code that has evolved since the last analysis.

Step 3: Reconciling Stale Files

Before introducing new data, CRG must purge obsolete entries. The _reconcile_stale_files() method (lines 108-124) scans the graph for files that are no longer present on disk or are now excluded by ignore patterns. Removing these stale entries first prevents phantom nodes from corrupting dependency queries during the subsequent parse phase.

Step 4: Finding Dependent Files

A changed file often impacts importers and downstream modules. The find_dependents() function (lines 86-104) traverses the existing graph using store.get_edges_by_target to locate files that import from or reference the changed paths. By default, CRG expands the change set up to two hops (configurable) to capture transitive dependencies.

The result is encapsulated in a DependentList, which is capped to prevent exponential growth in large monorepos.

Step 5: Building the Work List

The effective work list is the union of changed and dependent files, filtered to exclude ignored or binary paths. As implemented in the incremental_update function (lines 115-131), CRG performs an early-exit optimization: it computes a SHA-256 hash of each file’s contents and compares it against the hash stored in the existing graph node. Files whose hashes match are skipped entirely, avoiding redundant parsing.

Step 6: Parsing and Merging

Files requiring analysis are fed into the parsing pipeline. The entry point _parse_single_file() (lines 130-149) invokes language-specific parsers from code_review_graph/parser.py to extract nodes and edges.

CRG supports three execution strategies selected by _select_executor_kind:

• Serial: Used when CRG_SERIAL_PARSE=1 is set or for tiny repositories.
• ProcessPoolExecutor: Default for CPU-bound parsing of large codebases.
• ThreadPoolExecutor: Alternative for I/O-bound scenarios.

Each worker returns nodes, edges, and a content hash. The main thread then calls store.store_file_nodes_edges() to write these into the SQLite graph store.

Step 7: Persisting Deletions and Updating Metadata

To avoid foreign-key violations or nested-transaction errors, deletions are persisted before new insertions. The incremental_update routine (lines 190-199) calls store.remove_files_permanently() for any paths that disappeared from the filesystem.

After successful parsing, CRG commits the transaction and updates metadata (lines 200-207):

• last_updated timestamp
• last_build_type="incremental"
• VCS metadata (git_branch, git_head_sha)

Finally, language-specific resolvers (Python imports, ReScript cross-module references, Spring DI calls, etc.) are re-executed only if files of the corresponding language were processed (lines 210-233). This selective execution prevents unnecessary work and resolver failures on unrelated file types.

Usage Examples

Command-Line Incremental Update

Run an incremental update from the repository root:

crg update

Internally, this resolves to:

repo_root = find_repo_root()
store = GraphStore(get_db_path(repo_root))
result = incremental_update(repo_root, store)

Programmatic Control

Import the functions directly to customize behavior:

from pathlib import Path
from code_review_graph.graph import GraphStore
from code_review_graph.incremental import incremental_update, resolve_incremental_base

repo = Path("/path/to/my/repo")
store = GraphStore(get_db_path(repo))

# Explicitly override the base commit (optional)

base = resolve_incremental_base(repo, store) or "HEAD~1"

summary = incremental_update(repo, store, base=base)
print(f"Updated {summary['files_updated']} files")

Inspecting Change Impact

Audit what changed and which files depend on those changes:

from code_review_graph.incremental import get_changed_files, find_dependents
from code_review_graph.utils import normalize_file_path

changed = get_changed_files(repo, base="HEAD~1")
deps = set()

for path in changed:
    deps.update(find_dependents(store, normalize_file_path(repo / path)))

print("Changed:", changed)
print("Dependents:", deps)

Summary

• Base resolution: resolve_incremental_base() validates the previous commit SHA or triggers a full rebuild.
• Change detection: get_changed_files() uses git diff --name-status to identify modified paths.
• Dependency expansion: find_dependents() walks the graph up to two hops to find affected importers.
• Efficient parsing: Content hashing skips unchanged files; ProcessPoolExecutor enables parallel analysis.
• Atomic persistence: Stale files are removed via remove_files_permanently() before new nodes are stored.
• Metadata tracking: Updates to git_head_sha and last_build_type ensure the next incremental run has a valid baseline.

Frequently Asked Questions

What triggers a full rebuild instead of an incremental update in CRG?

CRG falls back to full_build() when resolve_incremental_base() cannot locate the stored git_head_sha in the current repository history, or when the C++ identity format version changes. This prevents applying incremental deltas on top of an incompatible or orphaned graph state.

How does CRG determine which files depend on a changed file?

The find_dependents() function queries the graph store using store.get_edges_by_target to reverse-lookup import relationships. It traverses up to two hops (configurable) to capture both direct importers and transitive dependents, returning a capped DependentList to control processing overhead.

Can CRG handle file renames and deletions during incremental updates?

Yes. get_changed_files() detects renames and deletions via Git’s name-status output. The _reconcile_stale_files() routine removes entries for deleted or ignored files before parsing begins, and store.remove_files_permanently() finalizes these deletions in the database before new nodes are inserted.

Is the incremental update process in CRG parallelized?

Yes. By default, CRG uses a ProcessPoolExecutor to parse files in parallel across CPU cores. It automatically selects ThreadPoolExecutor for I/O-bound workloads or falls back to serial execution when the CRG_SERIAL_PARSE=1 environment variable is set or the repository is small.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}