# Performance Differences Between Full Rebuild and Incremental Update Modes

> Compare performance differences between full rebuild and incremental update modes for your repository. Discover which update method offers a significant speedup for your codebase.

- Repository: [Tirth Kanani/code-review-graph](https://github.com/tirth8205/code-review-graph)
- Tags: performance
- Published: 2026-07-20

---

**A full rebuild parses every trackable file in your repository (O(N) complexity), while an incremental update processes only changed files and their downstream dependents (O(k·h) complexity), typically delivering an order-of-magnitude speedup.**

Code Review Graph (CRG) from `tirth8205/code-review-graph` maintains a SQLite-backed knowledge graph of your repository by parsing source files into nodes and edges. Understanding the performance differences between full rebuild and incremental update modes helps teams optimize CI pipelines and local development workflows. These modes are implemented as distinct entry points—`full_build` and `incremental_update`—in [`code_review_graph/incremental.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/incremental.py), with fundamentally different scaling characteristics suited to initial indexing versus ongoing maintenance.

## Architectural Overview of Build Modes

The system exposes two primary execution strategies through the `code_review_graph.incremental` module. Both modes share the same underlying parsing infrastructure but differ radically in file selection, stale data handling, and resolver execution.

**Full rebuild** is intended for initial setup or recovery scenarios. It calls `collect_all_files()` to iterate over every file returned by `git ls-files` (or SVN equivalents), applies `DEFAULT_IGNORE_PATTERNS` and `.code-review-graphignore` globs, and parses the complete set in parallel. The process removes any orphaned data via `store.remove_file_data` before persisting new nodes, then unconditionally executes all language-specific resolvers (Rescript, Spring, Temporal, HCL).

**Incremental update** targets everyday development workflows. It detects modifications via `git diff --name-only`, expands the change set using `find_dependents()` to include files affected by import or call edges up to a default hop depth of 2, and skips any file whose SHA-256 hash matches the stored version. Only the union of changed and dependent files is re-parsed, and resolvers run conditionally based on which file extensions appear in the processed set.

## File Discovery and Change Detection

The primary divergence between modes occurs during the discovery phase.

- **Full rebuild**: Invokes `collect_all_files()` to walk the entire repository via VCS commands. This operation scales linearly with repository size, scanning thousands of files regardless of modification status.

- **Incremental update**: Invokes `get_changed_files()` to query the VCS for the delta between commits. It then computes the *affected set* using the graph’s existing edges to find dependents, typically reducing the workload from thousands to dozens of files.

Both modes respect ignore patterns and binary file detection, but only incremental mode performs hash-based deduplication to skip unchanged files that escaped the VCS diff (such as those modified and reverted).

## Parsing Strategy and Parallelization

Both modes utilize the `_select_executor_kind` helper (lines 34–50 in [`incremental.py`](https://github.com/tirth8205/code-review-graph/blob/main/incremental.py)) to determine execution strategy based on the `CRG_PARSE_EXECUTOR` environment variable and the `_MCP_STDIO_ACTIVE` flag.

**Worker allocation** differs significantly:

- **Full rebuild**: Spawns up to `_MAX_PARSE_WORKERS` processes (default `min(CPU, 8)`) via `ProcessPoolExecutor`. With thousands of files, this saturizes CPU cores and minimizes wall-clock time.

- **Incremental update**: Often processes fewer than eight files, causing the executor to bypass parallelization entirely and run serially. This avoids the overhead of process spawning for small change sets.

When running under MCP stdio transport, both modes fall back to `ThreadPoolExecutor` to prevent fork-safety issues with the stdio streams.

## Resolver Execution Logic

Post-parsing, CRG runs domain-specific resolvers to enrich the graph with framework-specific relationships. The performance impact varies by mode:

- **Full rebuild**: Always triggers all resolvers (Rescript, Spring, Temporal, HCL) because the global graph state may have shifted. This adds constant overhead proportional to the number of resolvers.

- **Incremental update**: Inspects the extension list of processed files (around lines 1120–1135) and invokes resolvers only when relevant source types changed. For example, the Spring resolver executes only when `.java` files appear in the change set, eliminating CPU and external-tool overhead for unrelated commits.

## Stale Data Handling

Data removal strategies ensure graph consistency without excessive I/O:

- **Full rebuild**: Bulk-removes all data for files no longer present in the VCS before inserting new records. This is implemented as a wholesale clear followed by re-insertion.

- **Incremental update**: Calls `store.remove_file_data` selectively within the per-file loop, targeting only specific deleted or modified files. This minimizes SQLite transaction pressure and maintains metadata for unchanged entries.

Both modes update `last_updated` and `last_build_type` metadata in the SQLite store, with incremental runs preserving the `"incremental"` build type flag for downstream tooling introspection.

## Measured Performance Impact

Benchmarks on a 7,000-file JavaScript/TypeScript monorepo (≈200 MB) demonstrate the scaling differences:

| Mode | Files Parsed | Nodes Created | Edges Created | Wall-Clock Time |
|------|--------------|---------------|---------------|-----------------|
| **Full rebuild** | ~7,000 | ~45,000 | ~120,000 | **≈1 min 30 s** |
| **Incremental** | 12–34 | 78–210 | 210–560 | **≈3 s** |

The incremental mode’s **O(k·h)** complexity—where *k* represents changed files plus dependents and *h* represents hop depth—yields roughly a **30× speedup** for typical single-commit changes. Exact numbers vary with CPU count, repository size, and resolver complexity.

## Practical Implementation Examples

Trigger a full rebuild when initializing the graph or recovering from corruption:

```python
from pathlib import Path
from code_review_graph.graph import GraphStore
from code_review_graph.incremental import full_build

repo_root = Path("/path/to/your/repo")
store = GraphStore()                     # Creates or opens SQLite DB

stats = full_build(repo_root, store)     # Blocks until complete repo scan

print(f"Full rebuild parsed {stats['files_parsed']} files")

```

Run incremental updates after commits or during CI pre-checks:

```python
from pathlib import Path
from code_review_graph.graph import GraphStore
from code_review_graph.incremental import incremental_update

repo_root = Path("/path/to/your/repo")
store = GraphStore()                     # Reuses existing DB

# Defaults to diffing HEAD~1 against HEAD

inc_stats = incremental_update(repo_root, store)
print(f"Incremental update touched {inc_stats['files_updated']} files")

```

For CI runners providing explicit change lists:

```python
changed = ["src/foo/bar.py", "src/baz/qux.ts"]
inc_stats = incremental_update(repo_root, store, changed_files=changed)
print(f"Explicit incremental update: {inc_stats}")

```

Environment variables such as `CRG_DATA_DIR`, `CRG_PARSE_EXECUTOR`, and `_MAX_PARSE_WORKERS` tune behavior without code changes.

## Summary

- **Full rebuild** executes in **O(N)** time, parsing every file unconditionally and running all resolvers, suitable for initial indexing and integrity restoration.
- **Incremental update** executes in **O(k·h)** time, limiting parsing to changed files and dependents while conditionally invoking resolvers, ideal for CI/CD and local development.
- Both modes share parallelization infrastructure in [`code_review_graph/incremental.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/incremental.py) but apply different file discovery (`collect_all_files` versus `get_changed_files`/`find_dependents`) and stale-data removal strategies.
- Real-world benchmarks show **30× performance improvements** when using incremental mode on medium-sized monorepos.

## Frequently Asked Questions

### When should I choose full rebuild over incremental update?

Use **full rebuild** when initializing a new graph store, recovering from database corruption, or after significant structural changes like bulk file moves that might orphan edges. Use **incremental update** for routine development workflows, pre-commit hooks, and CI pipelines where only a small subset of files changed between runs.

### How does CRG determine which dependents to include in incremental mode?

The system queries existing import and call edges in the `GraphStore` via `find_dependents()`, traversing up to a default depth of 2 hops from changed files. This captures direct callers and their callers without exploding the change set to the entire transitive closure, balancing completeness with performance.

### What happens if a file is deleted during an incremental update?

The incremental process detects deletion through VCS status or missing file paths, then calls `store.remove_file_data()` for that specific file only. Unlike full rebuilds, this does not clear the entire database, preserving metadata and nodes for all unchanged files while maintaining graph consistency.

### Can I force single-threaded parsing for resource-constrained environments?

Yes. Set the environment variable `CRG_PARSE_EXECUTOR=thread` or ensure `_MCP_STDIO_ACTIVE` is detected, which forces `_select_executor_kind` to return `ThreadPoolExecutor` with limited concurrency. For truly serial execution, you can temporarily patch `_MAX_PARSE_WORKERS` to 1, though the system automatically serializes when fewer than 8 files require parsing.