How Incremental Parsing Works in code-review-graph: Workflow and Triggers

code-review-graph uses incremental parsing to update only changed files and their transitive dependents rather than rebuilding the entire knowledge graph from scratch, significantly reducing build times during development.

The tirth8205/code-review-graph repository implements a sophisticated incremental update system in code_review_graph/incremental.py. This module orchestrates change detection, dependency analysis, and selective re-parsing to maintain an accurate SQLite-backed graph store while minimizing redundant work.

The Incremental Parsing Workflow

The incremental parsing pipeline follows a deterministic sequence from change detection to metadata persistence. Each stage is designed to minimize the surface area of code requiring re-analysis.

Step 1: Detecting Repository Changes

The process begins with get_changed_files (line 70 in code_review_graph/incremental.py), which executes git diff --name-status -z <base> to enumerate added, modified, and deleted files since the last known commit. For SVN repositories, it falls back to svn status. This function returns a structured list that drives all downstream decisions.

Step 2: Resolving the Diff Base

When no explicit base commit is provided, resolve_incremental_base (line 44) queries the graph store for the stored git_head_sha metadata. If that commit SHA remains reachable in the repository history, it becomes the diff base; otherwise, the system automatically falls back to a full rebuild to ensure graph integrity.

Step 3: Reconciling Stale Files

Before parsing new changes, _reconcile_stale_files (line 9) purges graph entries for files that have been deleted or newly ignored via .code-review-graphignore. This prevents dangling nodes and ensures the graph accurately reflects the current working tree.

Step 4: Computing Transitive Dependencies

Modified files often impact code that imports or calls them. The find_dependents function (line 85) traverses import and call edges in the graph to discover files that depend on changed symbols. By default, this explores up to _MAX_DEPENDENT_HOPS levels of transitive dependencies, configurable to balance thoroughness versus performance.

Step 5: Parallel Parsing and Storage

The union of changed files, dependent files, and reconciled stale files forms the final parse list. Each file is processed by _parse_single_file (line 27), which invokes CodeParser.parse_bytes to extract nodes and edges using Tree-sitter. Parsing is parallelized via _make_executor using either process or thread pools based on _select_executor_kind heuristics.

Parsed data is written to the SQLite store via store_file_nodes_edges (lines 158–176), replacing outdated entries for those specific files while preserving unchanged portions of the graph.

Step 6: Metadata Update and Language Resolvers

Finally, _store_vcs_metadata updates the stored git_branch, git_head_sha, and CPP_IDENTITY_VERSION. If the C++ identity version has changed, this triggers a full rebuild guard condition. Language-specific resolvers—such as the Python import resolver or Spring DI resolver—execute conditionally only if files of their respective languages appear in the change set (lines 208–236).

What Triggers an Incremental Update?

Several conditions can initiate an incremental build, ranging from automated VCS monitoring to manual API invocation.

VCS Change Detection

The most common trigger occurs when get_changed_files detects non-empty output from git diff or svn status. Any addition, modification, or deletion since the last recorded commit automatically flags the repository for incremental processing.

Explicit API Invocation

Users or CI pipelines trigger updates directly via the Python API:

from pathlib import Path
from code_review_graph.graph import GraphStore
from code_review_graph.incremental import incremental_update

repo = Path("/path/to/repo")
store = GraphStore(Path("/tmp/graph.db"))

# Automatic incremental update based on git changes

result = incremental_update(repo, store)
print(f"Reparsed {result['files_updated']} files")

The base parameter allows targeted comparisons (e.g., base="v1.2.3"), while changed_files accepts a manual list to bypass VCS detection entirely.

Version Mismatches and Stale Entries

Two metadata conditions force rebuilds:

• C++ Identity Version: If the stored CPP_IDENTITY_VERSION constant differs from the current implementation (lines 71–79), the system executes a full rebuild before proceeding.
• Unreachable Commits: When resolve_incremental_base cannot locate the stored git_head_sha in the repository (e.g., after aggressive rebasing), it falls back to full_build rather than risk an inconsistent incremental state.

Dependency Propagation

Even when a file itself hasn't changed, updates to its dependencies trigger re-parsing. The find_dependents logic ensures that modifying a shared utility library invalidates and re-parses all consuming modules up to the configurable hop limit.

Implementation Example: Running Full and Incremental Builds

To initialize a new graph store or recover from corruption, use full_build. For daily development workflows, incremental_update provides efficient synchronization:

from code_review_graph.incremental import full_build, incremental_update

# First run or major refactoring: complete rebuild

full_build(repo, store)

# Daily development: incremental sync

result = incremental_update(
    repo, 
    store, 
    base="HEAD~1",  # Optional: specific comparison point

    changed_files=None  # Optional: override VCS detection

)

The function returns a result dict containing files_updated, allowing callers to log or validate the scope of changes processed.

Summary

• Incremental parsing in code-review-graph minimizes build times by re-parsing only changed files and their dependents.
• The workflow executes in code_review_graph/incremental.py through functions including get_changed_files, resolve_incremental_base, find_dependents, and _parse_single_file.
• Triggers include VCS diffs, manual API calls, unreachable base commits, and CPP_IDENTITY_VERSION mismatches.
• Dependency propagation uses graph edge traversal to find files impacted by changes to shared dependencies.
• Parallel parsing with automatic executor selection ensures efficient CPU utilization during updates.

Frequently Asked Questions

What happens if the stored base commit no longer exists?

If resolve_incremental_base cannot locate the stored git_head_sha in the repository (for example, after a force push or rebase), the system automatically falls back to full_build. This ensures graph consistency by rebuilding from scratch rather than attempting an incremental update against an unreachable history state.

How does code-review-graph handle file dependencies?

The find_dependents function (line 85) traverses import and call edges in the SQLite graph store to identify files that reference changed symbols. By default, it explores up to _MAX_DEPENDENT_HOPS levels of transitive dependencies, ensuring that modifying a utility file triggers re-parsing of all modules that import it, directly or indirectly.

Can I force a full rebuild instead of an incremental update?

Yes. Call full_build(repo, store) from code_review_graph/incremental.py to completely recreate the knowledge graph. This is recommended for initial indexing, after repository corruption, or when the C++ identity version changes and you want to ensure a clean state before resuming incremental updates.

What languages support incremental resolution?

The core incremental parsing engine is language-agnostic, but post-update resolvers run conditionally based on file extensions detected in the change set. According to the source code, specific implementations include a Python import resolver and a Spring DI resolver, with additional languages supported via the conditional execution blocks near lines 208–236 in incremental.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}