How Incremental Graph Updating Optimizes Performance in Code-Review-Graph

Incremental graph updating reduces parsing time from minutes to seconds by only reprocessing changed files and their dependents instead of rebuilding the entire repository graph.

Code-Review-Graph is an open-source tool that builds full-text knowledge graphs from source code repositories. Without optimization, every update would trigger a costly full rebuild. The incremental update system in code_review_graph/incremental.py solves this through targeted change detection, intelligent dependency tracing, and selective resolver invocation.

Detecting Only Changed Files

The first optimization step isolates exactly what needs work. The get_changed_files() function executes a git diff between the current state and the stored base commit, returning only the paths that actually changed.

# From code_review_graph/incremental.py lines 70-88

def get_changed_files(repo_root: Path, base: str) -> List[Path]:
    """Run git diff to find files changed since base commit."""
    result = subprocess.run(
        ["git", "diff", "--name-only", base, "HEAD"],
        cwd=repo_root,
        capture_output=True,
        text=True
    )
    return [repo_root / p for p in result.stdout.strip().split("\n") if p]

This avoids the O(n) cost of scanning all tracked files, where n is the repository size.

Reusing the Previous Build as a Diff Base

The resolve_incremental_base() function locates the optimal starting point for comparison. It reads the git_head_sha stored in graph metadata and validates that the commit still exists in the repository (handling rebase and force-push scenarios).

# From code_review_graph/incremental.py lines 44-68

def resolve_incremental_base(repo_root: Path, store: GraphStore) -> Optional[str]:
    """Determine the base commit for incremental diffing."""
    stored_sha = store.get_metadata("git_head_sha")
    if not stored_sha:
        return None
    
    # Verify commit still exists (may have been rebased)

    result = subprocess.run(
        ["git", "cat-file", "-t", stored_sha],
        cwd=repo_root,
        capture_output=True
    )
    return stored_sha if result.returncode == 0 else None

When no valid base exists—such as on first run—the system gracefully falls back to a full build.

Finding Impacted Dependents

Changed files ripple through the graph. The find_dependents() function walks upstream dependencies to locate all nodes potentially affected by a modification.

Key implementation details from code_review_graph/incremental.py lines 85-100:

• Direct dependents: files that import or reference the changed file
• Transitive dependents: files dependent on those dependents, up to a configurable hop limit
• Safety cap: expansion stops at 500 files maximum to prevent runaway operations

This bounds the update scope to the blast radius of the change rather than the entire codebase.

Skipping Unchanged Files with Content Hashing

Before parsing any file, the system verifies whether work is actually needed. The incremental logic computes a content hash and compares it against stored metadata.

# Conceptual flow from code_review_graph/incremental.py lines 44-51

current_hash = hash_file_content(file_path)
stored_nodes = store.get_nodes_by_file(file_path)

if stored_nodes and stored_nodes[0].content_hash == current_hash:
    # File unchanged — reuse existing graph nodes

    continue

This eliminates redundant parsing when file modifications are reverted or when git reports changes to line endings only.

Cleaning Up Stale Data

Removed or ignored files would otherwise leave orphaned nodes in the graph. The _reconcile_stale_files() function prunes these entries:

# From code_review_graph/incremental.py lines 13-22

def _reconcile_stale_files(store: GraphStore, current_files: Set[Path]) -> int:
    """Remove graph entries for files that no longer exist or are now ignored."""
    stale = store.get_all_file_paths() - current_files
    for path in stale:
        store.remove_nodes_by_file(path)
    return len(stale)

This prevents unbounded graph growth and ensures query results remain accurate.

Selective Resolver Reruns

Language-specific resolvers perform heavyweight cross-module analysis. The incremental system tracks which languages actually changed and invokes only the relevant resolvers.

From code_review_graph/incremental.py lines 108-124:

changed_languages = detect_languages(changed_files)

if "python" in changed_languages:
    PythonResolver(store).resolve()
if "rescript" in changed_languages:
    ReScriptResolver(store).resolve()

# Spring, etc. — skipped entirely if no files of that type changed

This avoids O(m²) import resolution work when m modules are unchanged.

Performance Impact

Metric	Full Build	Incremental Update
Files parsed	All tracked files (thousands)	Changed + dependents (tens to hundreds)
Parsing time	Minutes for large repositories	Seconds to sub-minute
Resolver work	Complete graph analysis	Affected languages only
Disk I/O	Reads every source file	Reads modified files only

The result is linear-time scaling with change size rather than repository size.

Complete Incremental Update Example

from pathlib import Path
from code_review_graph.graph import GraphStore
from code_review_graph.incremental import incremental_update, resolve_incremental_base

# 1. Open the graph database

repo_root = Path("/path/to/your/repo")
store = GraphStore(path=repo_root)

# 2. Auto-detect or specify diff base

base = resolve_incremental_base(repo_root, store)  # Returns None on fresh clone

# 3. Execute incremental update

result = incremental_update(
    repo_root=repo_root,
    store=store,
    changed_files=None,      # Auto-detect via git

    reconcile_stale=True,    # Clean up removed files

)

print(f"Files re-parsed: {result['files_updated']}")
print(f"New nodes: {result['total_nodes']}")
print(f"New edges: {result['total_edges']}")

Summary

• Change detection via git diff eliminates work on unmodified files
• Stored commit SHA with validation enables reliable incremental bases through rebases
• Dependent expansion with caps contains the update to affected regions
• Content hashing prevents redundant parsing of identical files
• Stale entry cleanup maintains graph accuracy and bounded size
• Language-gated resolvers avoid expensive cross-module analysis for unchanged code

These mechanisms together transform Code-Review-Graph from a batch tool into a responsive system suitable for interactive development and continuous integration.

Frequently Asked Questions

What triggers a full rebuild instead of an incremental update?

A full rebuild occurs when resolve_incremental_base() returns None. This happens on first run (no stored git_head_sha), when the stored commit has been garbage-collected or force-pushed away, or when explicitly requested by passing base=None to incremental_update().

How does the system handle force-push or rebase operations?

The stored git_head_sha is validated with git cat-file -t before use. If the commit no longer exists, resolve_incremental_base() returns None, triggering either a full rebuild or allowing the caller to specify an alternative base such as HEAD~10.

What is the 500-file limit on dependent expansion?

The find_dependents() function stops traversing the dependency graph after 500 files to prevent performance degradation on changes with extremely large blast radios (such as modifications to core utility modules). This cap is configurable and logs a warning when reached.

Can incremental updates miss cross-file relationships?

No. The dependency walker traces both direct and transitive dependents, and content hashing catches semantic changes even when git reports a file as modified. Language resolvers re-run whenever any file of that language changes, ensuring import relationships and symbol references stay current.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}