How code-review-graph Achieves Incremental Updates in Under 2 Seconds

code-review-graph performs sub-2-second incremental updates by processing only changed files and their direct dependents—typically ten or fewer files—while using SHA-256 hashing to skip unchanged content and parallelizing parsing across worker pools.

The tirth8205/code-review-graph repository implements a high-performance incremental update pipeline in code_review_graph/incremental.py that rebuilds only the necessary portions of its code-analysis graph. By avoiding full repository rescans and leveraging dependency-aware selective reprocessing, the system consistently completes updates in under two seconds even on large codebases.

Detecting Changes and Filtering Noise

The incremental workflow begins by identifying exactly what changed since the last analysis. The incremental_update function calls get_changed_files(repo_root, base) to execute a git diff (or svn status for Subversion repositories) and retrieve the list of modified files.

Before processing these files, the system filters out irrelevant paths. The _load_ignore_patterns helper reads both the built-in DEFAULT_IGNORE_PATTERNS and repository-specific .code-review-graphignore files to exclude directories like node_modules and generated assets. This ensures the pipeline touches only source files that matter for code analysis.

Dependency-Aware Reprocessing

Simply reparsing changed files is insufficient for maintaining graph accuracy. The system must also identify files that import from or call into modified modules. For every changed file, incremental_update invokes find_dependents(store, full_path), which traverses the graph’s import- and call-edges up to two hops by default.

The resulting changed_files ∪ dependent_files union—stored in the all_files variable—represents the minimal set requiring reprocessing. In practice, this set rarely exceeds ten files, even in large repositories.

Content Hashing to Skip Unchanged Files

To avoid redundant parsing of files that appear in the dependency set but whose content remains identical, the system computes a SHA-256 hash of each candidate file. In the hash-compare block at lines 104–110 of incremental.py, the code checks if existing_nodes against the stored hash. When the hashes match, the file is excluded from the reprocessing queue, further reducing workload.

Parallel Parsing and Store Updates

Files that survive the hashing filter are dispatched to a worker pool for parallel processing. The _make_executor(_MAX_PARSE_WORKERS) factory chooses between ThreadPoolExecutor (on Windows or when MCP stdio is active) and ProcessPoolExecutor (otherwise), then maps _parse_single_file across the candidate set using executor.map.

Each successfully parsed file updates the graph store via store.store_file_nodes_edges, which atomically replaces the previous nodes and edges for that file. After all files process, the system refreshes metadata including last_updated timestamps and VCS information.

Selective Resolver Passes

Language-specific post-processing resolvers for ReScript, Spring, Temporal, and HCL execute only when files of the corresponding type appear in the change set. Conditional checks like if rescript_changed prevent unnecessary computation, keeping the update bounded to the actual scope of changes.

Debounced Watch Mode

When running in continuous watch mode, the watch function batches file-system events using a 300-millisecond debounce (_DEBOUNCE_SECONDS = 0.3). This prevents cascading updates during rapid save sequences and ensures each processing batch remains well under the two-second target.

# Trigger an incremental update from Python

from pathlib import Path
from code_review_graph.incremental import incremental_update
from code_review_graph.graph import GraphStore

repo = Path("/path/to/my/repo")
store = GraphStore()
result = incremental_update(repo, store)  # base defaults to HEAD~1

print("Updated files:", result["files_updated"])

# Enable auto-updating watch mode

from code_review_graph.incremental import start_watch_thread

store = GraphStore()
watch_thread = start_watch_thread(Path("."), store)

# Background thread reflects edits after each 300ms debounce

Summary

• Minimal file set: Only changed files and their dependents (up to two hops) enter the processing queue.
• Content-aware skipping: SHA-256 hashing prevents reprocessing of unchanged files that appear in dependency chains.
• Parallel execution: Worker pools leverage multiple cores to parse the small candidate set rapidly.
• Selective resolvers: Language-specific post-processors run only when relevant file types change.
• Debounced watching: 300-millisecond batching prevents update cascades in continuous monitoring scenarios.

Frequently Asked Questions

How does the system determine which files to reparse?

The system calls get_changed_files to identify modified files via git diff, then uses find_dependents to traverse import and call edges up to two hops in the existing graph. It merges these sets and filters against SHA-256 hashes to exclude files whose content matches the stored version.

What prevents the watch mode from triggering too frequently?

The watch function implements a 300-millisecond debounce (_DEBOUNCE_SECONDS = 0.3) that batches rapid file-system events into single update cycles. This prevents the system from initiating new incremental updates before the previous one completes.

Does incremental updating work with all programming languages?

Yes, the core incremental pipeline is language-agnostic. However, language-specific resolvers for ReScript, Spring, Temporal, and HCL execute selectively only when files of those types are modified, ensuring no unnecessary post-processing overhead for other languages.

How does the system handle transitive dependencies across multiple hops?

By default, find_dependents traverses up to two hops in the dependency graph. This captures most real-world coupling patterns while keeping the reprocessing set small. If a file deeper in the chain changes, the next incremental update will pick it up when its immediate dependents trigger the cascade.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}