How Incremental Graph Updating Works in code-review-graph: A 10-Step Pipeline

Incremental graph updating in code-review-graph rebuilds only changed files and their dependents by diffing against the last stored commit, walking import/call edges up to two hops, and re-parsing in parallel while skipping unchanged content via SHA-256 hashing.

The code-review-graph library constructs a knowledge graph mapping symbols and relationships across a codebase. Re-parsing the entire repository after every change is prohibitively expensive for large projects, so the library implements an incremental graph updating mechanism that minimizes CPU and I/O overhead while maintaining accuracy. This process is orchestrated in code_review_graph/incremental.py through a deterministic pipeline that detects changes, expands the affected dependency cone, and updates the graph store transactionally.

The Incremental Update Pipeline

The incremental_update() function in code_review_graph/incremental.py executes a ten-step workflow to synchronize the graph with the working directory. Each step targets a specific concern, from VCS interaction to parallel parsing.

Step 1: Repository Root Detection

The system first locates the project boundary by walking up the directory tree searching for a .git or SVN marker. The find_repo_root() and find_project_root() functions handle this discovery, though operators can override the result by setting the CRG_REPO_ROOT environment variable. This ensures the subsequent git diff commands execute within the correct repository context.

Step 2: Diff Base Resolution

To determine what constitutes a "change," the library invokes resolve_incremental_base() to identify the last successfully processed commit. It retrieves the stored git_head_sha from the graph metadata; if this SHA still exists in the repository, it serves as the diff base. If the SHA is unreachable (due to a history rewrite or fresh clone), the system falls back to a full rebuild to guarantee consistency.

Step 3: Changed File Discovery

Using the resolved base, the system executes git diff --name-status -z <base> (or the SVN equivalent) via get_changed_files(). The _decode_name_status_paths() helper parses the null-delimited output, ensuring that both the old and new paths of renamed files are captured for processing. This step produces the initial set of files requiring attention.

Step 4: Stale File Reconciliation

Before adding new data, the system removes orphaned graph elements. The _reconcile_stale_files() function identifies files that have been deleted from the repository or are now excluded by ignore patterns, purging their associated nodes and edges from the store. This prevents phantom relationships from polluting downstream analysis.

Step 5: Dependent File Resolution

Changes propagate through import and call relationships. The find_dependents() function traverses the existing graph's edges to collect files that directly or indirectly depend on the changed set. The walk uses _single_hop_dependents() iteratively, bounded by _MAX_DEPENDENT_HOPS (defaulting to 2) and capped at _MAX_DEPENDENT_FILES (500) to prevent combinatorial explosions in highly interconnected codebases.

Step 6: Filtering and Hash Verification

Not every file in the changed or dependent set requires parsing. The system filters out ignored and binary files, then performs a SHA-256 hash comparison against stored node hashes. If a file's content hash matches the existing record—even if Git reports it as modified (e.g., due to a timestamp change)—the parser skips it, eliminating redundant work.

Step 7: Parallel File Parsing

The remaining files are distributed across a worker pool for parsing. The _make_executor() function selects between process and thread pools based on the runtime context; it defaults to process-based isolation for CPU-bound parsing but falls back to threads when running under FastMCP stdio transport to avoid deadlocks on Windows. Each worker invokes _parse_single_file(), reusing a cached CodeParser instance to amortize initialization costs.

Step 8: Graph Store Updates

Parsed nodes and edges are persisted through store_file_nodes_edges(). To avoid foreign-key violations in the SQLite backend, deletions are committed before insertions within the same transaction. This ensures the graph remains internally consistent even if the update process is interrupted.

Step 9: Metadata Persistence

After a successful update, _store_vcs_metadata() records the current last_updated timestamp, last_build_type, branch name, and commit SHA via store.set_metadata(). These values become the diff base for the next incremental run, creating a continuous synchronization chain.

Step 10: Language-Specific Resolvers

Finally, the system invokes only the resolvers for languages that actually changed. Helper functions like _run_python_resolver(), _run_rescript_resolver(), or _run_spring_resolver() execute language-specific post-processing (such as resolving Spring bean references or Temporal workflow calls) without wasting cycles on unaffected language domains.

Performance Safeguards and Optimizations

The incremental system incorporates several defensive mechanisms to ensure predictable performance:

• Bounded Dependency Walking: The _MAX_DEPENDENT_HOPS limit prevents the system from re-parsing the entire graph when a low-level utility file changes, while _MAX_DEPENDENT_FILES acts as a hard ceiling on the work queue.

• Content-Addressed Caching: SHA-256 hashing ensures that file renames without content changes, or mode changes without content changes, do not trigger parser invocations.

• Adaptive Concurrency: The _select_executor_kind() logic automatically chooses between ProcessPoolExecutor and ThreadPoolExecutor based on the transport mechanism, balancing throughput against compatibility constraints.

Practical Implementation

To perform an incremental update in your own tooling, initialize a GraphStore and invoke the update function:

from pathlib import Path
from code_review_graph.graph import GraphStore
from code_review_graph.incremental import incremental_update, full_build

# Initialize a GraphStore (SQLite backed)

store = GraphStore(Path("/tmp/myrepo/graph.db"))

# Full rebuild – usually done once after cloning a repo

full_build(Path("/tmp/myrepo"), store)

# Later, perform an incremental update (auto-detects changed files)

result = incremental_update(Path("/tmp/myrepo"), store)

print("Files re-parsed:", result["files_updated"])
print("New nodes added:", result["total_nodes"])
print("New edges added:", result["total_edges"])

You can also force an update against a specific baseline, bypassing automatic diff detection:

incremental_update(
    repo_root=Path("/tmp/myrepo"),
    store=store,
    base="v1.2.3",               # any git ref that resolves to a commit

    changed_files=None,          # let the function compute diffs

    reconcile_stale=True,
)

Summary

• Change detection relies on git diff against the last stored commit SHA, with automatic fallback to full builds when history is unreachable.
• Impact analysis walks import and call edges up to _MAX_DEPENDENT_HOPS (default 2) to find affected files, capped at 500 files.
• Efficiency is achieved through SHA-256 content hashing that skips unchanged files and parallel parsing using process pools.
• Consistency is maintained by reconciling stale files before updates and persisting deletions before insertions in the graph store.
• Extensibility allows language-specific resolvers to run only for languages present in the changed set.

Frequently Asked Questions

How does code-review-graph handle renamed files during incremental updates?

The get_changed_files() function in code_review_graph/incremental.py uses git diff --name-status -z and _decode_name_status_paths() to capture both the old and new paths of renamed files. Both paths enter the dependency analysis, ensuring that edges pointing to the old filename are removed and edges to the new filename are established.

What happens if the last stored commit no longer exists in the repository?

When resolve_incremental_base() cannot locate the stored git_head_sha (for example, after a rebase or force-push), the system abandons the incremental path and triggers a full rebuild. This guarantees that the graph does not retain stale references from a diverged history.

Can I limit how many dependent files are re-parsed when a core library changes?

Yes. The system respects two constants in code_review_graph/incremental.py: _MAX_DEPENDENT_HOPS (default 2) limits how many levels of imports or calls to traverse, and _MAX_DEPENDENT_FILES (default 500) caps the total number of files added to the parsing queue. These bounds prevent a single change to a utility module from triggering a full repository re-parse.

Does incremental updating work with version control systems other than Git?

The codebase supports SVN alongside Git. The get_changed_files() function detects the repository type and executes the appropriate diff command (or SVN equivalent) to discover changed files. However, the metadata storage and base resolution logic primarily targets Git workflows, with SVN support handling basic diff scenarios.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}