How Incremental Update Finds and Re‑Parses Only Changed Files in Code Review Graph

The incremental_update function in code_review_graph/incremental.py performs a targeted graph rebuild by detecting VCS changes, traversing import edges to find dependents, filtering unchanged files via SHA‑256 hashes, and re‑parsing only the minimal necessary subset.

The tirth8205/code-review-graph repository implements an efficient incremental update mechanism that avoids full rebuilds after every code change. By leveraging version control integration and graph dependency tracking, the system identifies exactly which files require re‑parsing to keep the code‑review graph synchronized with the repository state.

Detecting Changes Through VCS Integration

The incremental process begins with change detection inside incremental_update. If the caller does not provide an explicit list of changed files, the function invokes get_changed_files to query the version control system directly.

For Git repositories, this executes git diff --name‑only -z to retrieve null‑terminated paths of added, modified, or removed files. Subversion repositories use equivalent svn status or svn diff commands. All discovered paths pass through the repository's ignore rules, which combine .code‑review‑graphignore patterns with built‑in defaults, as implemented around line 68 of incremental.py.

Locating Dependent Files via Graph Traversal

Once changed files are identified, the system determines which other files might be affected by calling find_dependents for each changed path. This function traverses the graph's import and call edges to discover files that import from or otherwise depend on the changed modules.

The traversal respects configurable safety limits to prevent performance degradation in large monorepos. The _MAX_DEPENDENT_HOPS parameter defaults to 2 hops and caps discovery at 500 files, as defined near line 645 in incremental.py. Discovered dependent paths are normalized back to repository‑relative paths before processing continues.

Hash‑Based Deduplication and Pruning

Before parsing begins, the system performs a final optimization pass to eliminate redundant work. The union of changed and dependent files forms the candidate set all_files.

Each candidate undergoes two validation checks:

• Ignore pattern validation via _should_ignore, respecting both user‑defined and built‑in exclusion patterns
• Content hash comparison using SHA‑256, where the current file hash is compared against the hash stored in the graph database

If hashes match, the file is skipped entirely, as implemented near line 1045. Deleted files trigger store.remove_file_data and are purged from the graph before any parsing commences.

Selective Re‑Parsing Execution

The remaining files—those with changed content or changed dependencies—undergo targeted parsing. The system selects execution strategy via _select_executor_kind:

• Parallel processing using process or thread pools (default for most repositories)
• Serial execution when the environment variable CRG_SERIAL_PARSE=1 is set or for very small repositories

Each file is processed through CodeParser.parse_bytes, with resulting nodes and edges written back via store.store_file_nodes_edges. After completion, the graph metadata—including timestamps, build type, and VCS information—is refreshed and committed.

Practical Implementation Examples

The following examples demonstrate full builds versus incremental updates:

from pathlib import Path
from code_review_graph.graph import GraphStore
from code_review_graph.incremental import incremental_update, full_build

# Full rebuild (typically run once per repository)

repo_root = Path("/path/to/repo")
store = GraphStore(Path("/tmp/graph.db"))
full_build(repo_root, store)

# Incremental update after a commit

# Automatically discovers changed files via `git diff HEAD~1`

result = incremental_update(repo_root, store)

print("Files updated:", result["files_updated"])
print("Changed:", result["changed_files"])
print("Dependents:", result["dependent_files"])

Pass an explicit list of changed files when running in CI environments:

changed = ["src/module_a.py", "tests/test_b.py"]
result = incremental_update(repo_root, store, changed_files=changed)

Enable the filesystem watcher for automatic updates:

from code_review_graph.incremental import start_watch_thread

thread = start_watch_thread(repo_root, store)

# The thread calls `incremental_update` whenever files change

Summary

• VCS‑driven detection ensures only repository‑changed files enter the candidate set via get_changed_files and git diff/svn status
• Graph traversal discovers affected dependents through find_dependents with configurable hop limits (default 2, max 500 files)
• Hash verification prevents re‑parsing unchanged content using SHA‑256 comparison near line 1045
• Targeted parsing processes only the union of changed files and their dependents, using parallel execution by default
• Cleanup handling removes deleted files via store.remove_file_data before parsing begins

Frequently Asked Questions

How does the system handle files that haven't changed but import modified modules?

The find_dependents traversal identifies these files as dependents, and they are included in the re‑parse set even if their SHA‑256 hash matches the stored version. This ensures import and call edges are updated to reflect changes in the modules they depend on, maintaining graph accuracy.

What version control systems are supported for automatic change detection?

The get_changed_files function supports Git via git diff --name‑only -z and Subversion via svn status or svn diff commands. The system auto‑detects the VCS type based on repository metadata.

Can I bypass the hash check to force re‑parsing specific files?

While the API does not expose a direct "force re‑parse" flag, you can trigger re‑parsing by modifying the file content or by removing the stored hash entry from the GraphStore database before calling incremental_update.

How do I enable serial parsing instead of parallel execution?

Set the environment variable CRG_SERIAL_PARSE=1 before running the update. Alternatively, the system automatically selects serial execution for very small repositories where parallelization overhead would exceed the performance benefit.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}