How Incremental Graph Updating Works in code-review-graph: A Technical Deep Dive

Incremental graph updating in code-review-graph minimizes rebuild costs by diffing the repository against the last stored commit, re-parsing only changed files and their transitive dependents, and applying updates transactionally to the knowledge graph.

The code-review-graph library maintains a persistent knowledge graph representing symbols and their relationships across a source repository. Rather than performing a full rebuild after every modification, the library implements an incremental graph updating strategy that targets only changed files and their dependencies. This approach, implemented primarily in code_review_graph/incremental.py, dramatically reduces CPU and I/O overhead while maintaining graph accuracy.

Change Detection and Base Resolution

The process begins by establishing context for what constitutes "changed."

Locating the Repository Root

The system calls find_repo_root() and find_project_root() to walk up the directory tree searching for .git or SVN markers. Users can override this discovery by setting the CRG_REPO_ROOT environment variable, ensuring the incremental updater operates on the correct source tree regardless of the current working directory.

Selecting the Diff Base

The resolve_incremental_base() function retrieves the last successfully stored commit SHA (git_head_sha) from the graph metadata. If this commit still exists in the repository, it serves as the base for comparison; otherwise, the system falls back to a full rebuild to ensure consistency.

Discovering and Filtering Changed Files

Once the base is established, the system identifies exactly what has changed.

Git Diff Execution

The get_changed_files() function executes git diff --name-status -z <base> (or the SVN equivalent) and processes the output through _decode_name_status_paths(). This captures not only modified files but also handles renames correctly by tracking both the old and new paths in a single operation.

Dependency Expansion

Re-parsing only the directly changed files would miss semantic updates in dependent modules. The find_dependents() function traverses the existing graph's import and call edges using _single_hop_dependents() to collect files that depend on the changed set. To prevent combinatorial explosions, the traversal respects two critical limits:

• _MAX_DEPENDENT_HOPS: Default of 2 hops maximum
• _MAX_DEPENDENT_FILES: Hard cap of 500 files

Content-Based Deduplication

Even if a file appears in the changed list (for example, due to a rename), the system compares SHA-256 hashes against stored node data to skip files whose contents remain identical, avoiding unnecessary parser invocations.

Reconciliation and Parsing

Before parsing new data, the system cleans up stale entries and then processes the affected files.

Stale File Removal

The _reconcile_stale_files() function removes nodes and edges for files that no longer exist in the repository or are now excluded by ignore patterns, ensuring the graph does not reference deleted symbols.

Parallel File Parsing

The _parse_single_file() worker function processes files using cached CodeParser instances for performance. The _select_executor_kind() helper chooses between process pools (default for isolation) and thread pools (used when FastMCP stdio transport is active to prevent Windows deadlocks). Files are parsed either serially or in parallel depending on the executor configuration.

Graph Store Updates

Parsed nodes and edges are persisted via store_file_nodes_edges(). Importantly, deletions are committed before insertions to avoid nested transaction errors in the SQLite-backed GraphStore.

Metadata Recording and Resolver Execution

After the core graph update, the system finalizes state for future incrementality.

VCS Metadata Persistence

The _store_vcs_metadata() function records the current branch name, commit SHA, and timestamp via store.set_metadata(), updating last_updated, last_build_type, and git_head_sha so subsequent incremental updates have a reliable base.

Language-Specific Resolution

Optionally, the system invokes _run_*_resolver() helpers only for languages that actually changed (Python, ReScript, Spring, Temporal, HCL, etc.). This selective execution avoids the cost of running resolvers for unaffected language ecosystems.

Practical Implementation Examples

To perform an incremental update after initializing a graph store:

from pathlib import Path
from code_review_graph.graph import GraphStore
from code_review_graph.incremental import incremental_update, full_build

# Initialise a GraphStore (SQLite backed)

store = GraphStore(Path("/tmp/myrepo/graph.db"))

# Full rebuild – usually done once after cloning a repo

full_build(Path("/tmp/myrepo"), store)

# Later, perform an incremental update (auto-detects changed files)

result = incremental_update(Path("/tmp/myrepo"), store)

print("Files re-parsed:", result["files_updated"])
print("New nodes added:", result["total_nodes"])
print("New edges added:", result["total_edges"])

You can also specify an explicit diff base, such as a tag or SHA:

incremental_update(
    repo_root=Path("/tmp/myrepo"),
    store=store,
    base="v1.2.3",               # any git ref that resolves to a commit

    changed_files=None,          # let the function compute diffs

    reconcile_stale=True,
)

Summary

• Change detection relies on git diff against the last stored commit SHA, with automatic fallback to full rebuilds when the base is unreachable.
• Dependency expansion uses the existing graph to find transitive dependents, bounded by _MAX_DEPENDENT_HOPS (2) and _MAX_DEPENDENT_FILES (500) to control scope.
• Content hashing prevents re-parsing files whose contents are unchanged despite VCS appearances.
• Parallel execution adapts to the runtime environment, using process pools by default and thread pools for FastMCP compatibility.
• Transactional updates delete stale data before inserting new nodes/edges to maintain SQLite consistency.
• Metadata tracking records the current commit and branch to enable reliable incremental bases for subsequent updates.

Frequently Asked Questions

How does code-review-graph determine which files need re-parsing?

The system runs git diff --name-status -z via get_changed_files() in code_review_graph/incremental.py to identify files modified since the last stored git_head_sha. It then expands this set using find_dependents() to include files that import or call the changed modules, bounded by configurable hop limits.

What prevents the dependency traversal from including too many files?

Two constants in incremental.py constrain the search: _MAX_DEPENDENT_HOPS defaults to 2 levels of transitive dependencies, and _MAX_DEPENDENT_FILES caps the total at 500 files. These limits prevent a single change in a core utility from triggering a full repository re-parse.

How are file renames handled during incremental updates?

The _decode_name_status_paths() function parses Git's rename detection output (indicated by R status), capturing both the original and new file paths. This ensures the system removes nodes for the old path and creates them for the new path while preserving relationships if the content hash remains unchanged.

Can I force a full rebuild instead of an incremental update?

Yes. The full_build() function in code_review_graph/incremental.py bypasses all diff logic and re-parses the entire repository. This is typically run once after initial clone or when the incremental metadata becomes inconsistent (for example, after the base commit is garbage collected).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}