# How the Incremental Engine in code-review-graph Detects File Changes Using SHA-256 Hashes

> Discover how the incremental engine in code-review-graph uses SHA-256 hashes to efficiently detect file changes, saving time by skipping unchanged files.

- Repository: [Tirth Kanani/code-review-graph](https://github.com/tirth8205/code-review-graph)
- Tags: internals
- Published: 2026-07-26

---

**The incremental engine combines VCS diffing with SHA-256 checksums to skip parsing files whose byte content remains identical, avoiding redundant analysis work.**

The `code-review-graph` repository implements a high-performance incremental analysis system that minimizes re-parsing overhead in large codebases. When processing repository updates, the engine located in [`code_review_graph/incremental.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/incremental.py) employs a two-stage filtering mechanism that first identifies candidate files through version control, then verifies actual content changes using cryptographic hashing before invoking the parser.

## Two-Stage Change Detection Process

The incremental update workflow relies on distinct phases to narrow the scope of files requiring re-analysis.

### Stage 1: VCS-Based Candidate Selection

The engine begins by querying the version control system to establish an initial set of potentially modified files. The `get_changed_files()` function executes `git diff --name-only -z` (or SVN equivalent commands) to retrieve files changed since a reference commit (default `HEAD~1`). 

Additionally, the system calculates **dependent files**—modules that import from changed files—to ensure structural relationship updates propagate through the graph. This logic appears in [`incremental.py`](https://github.com/tirth8205/code-review-graph/blob/main/incremental.py) lines 1035-1044:

```python
dependent_files: set[str] = set()
for rel_path in changed_files:
    full_path = str(repo_root / rel_path)
    deps = find_dependents(store, full_path)
    for d in deps:
        try:
            dependent_files.add(str(Path(d).relative_to(repo_root)))
        except ValueError:
            pass

```

### Stage 2: SHA-256 Content Verification

For every candidate file (changed files plus dependents), the engine performs a **quick hash check** to eliminate false positives. As implemented in [`incremental.py`](https://github.com/tirth8205/code-review-graph/blob/main/incremental.py) lines 1045-1051, the system reads raw bytes, computes a SHA-256 digest, and compares it against the stored hash in the graph database:

```python

# Quick hash check to skip unchanged files

try:
    raw = abs_path.read_bytes()
    fhash = hashlib.sha256(raw).hexdigest()
    existing_nodes = store.get_nodes_by_file(str(abs_path))
    if existing_nodes and existing_nodes[0].file_hash == fhash:
        continue
except (OSError, PermissionError):
    pass
to_parse.append(rel_path)

```

When the computed `fhash` matches `existing_nodes[0].file_hash`, the file is skipped entirely. Only files with mismatched hashes proceed to `to_parse` for actual AST analysis.

## SHA-256 Implementation Details

### Core Hashing Logic in incremental.py

The inline hashing uses Python's standard `hashlib` module for deterministic, platform-independent checksums. The `hexdigest()` output provides a 64-character string suitable for SQLite storage and comparison. This approach executes for every candidate file during the `incremental_update()` workflow.

### Reusable Utility in parser.py

The codebase also exposes a dedicated utility function in [`code_review_graph/parser.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/parser.py) lines 1980-1982 for standalone hash generation:

```python
def file_hash(path: Path) -> str:
    """SHA-256 hash of file contents."""
    return hashlib.sha256(path.read_bytes()).hexdigest()

```

This function serves other components requiring content-addressed caching or duplicate detection outside the incremental engine.

## Database Integration and Storage

The `file_hash` value persists in the graph store's `nodes` table, associated with each parsed file node. When `store_file_nodes_edges()` commits new analysis results during both full builds and incremental updates, it stores the current SHA-256 value alongside AST metadata. The `store.get_nodes_by_file()` method retrieves these historical hashes for comparison during subsequent runs.

## Practical Performance Benefits

This dual-filter architecture delivers significant optimizations:

- **VCS noise reduction**: File permission changes, timestamp updates, or rename operations that preserve content no longer trigger expensive re-parsing.
- **Dependency accuracy**: The system correctly handles cases where dependent files remain semantically unchanged despite imports being modified.
- **Cryptographic certainty**: SHA-256 collisions are computationally infeasible, ensuring the engine never incorrectly skips genuinely modified files.

## Summary

- The incremental engine in [`code_review_graph/incremental.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/incremental.py) uses **SHA-256 hashing** to verify file content changes after initial VCS diffing.
- The hash comparison occurs at lines 1045-1051, using `hashlib.sha256(raw).hexdigest()` and checking against `existing_nodes[0].file_hash`.
- A reusable `file_hash()` utility exists in [`code_review_graph/parser.py`](https://github.com/tirth8205/code-review-graph/blob/main/code_review_graph/parser.py) for consistent checksum generation.
- Only files with mismatched SHA-256 hashes enter the `to_parse` queue, eliminating redundant AST construction.
- The mechanism handles VCS false positives, permission changes, and content-preserving renames without unnecessary processing overhead.

## Frequently Asked Questions

### Why does the engine use SHA-256 instead of faster hashes like MD5?

**SHA-256 provides collision resistance required for build system reliability.** While MD5 offers marginally faster computation, the incremental engine prioritizes correctness guarantees over micro-optimizations, ensuring that two different file contents never produce identical hash values that would cause the system to skip necessary re-parsing.

### What happens when a file exists in the repository but has no stored hash?

**The engine treats missing database entries as changed files requiring parsing.** If `store.get_nodes_by_file()` returns an empty list (`existing_nodes` is falsy), the hash comparison is bypassed and the file automatically proceeds to the `to_parse` list for initial AST analysis and hash storage.

### Does the SHA-256 check handle binary files or permission errors?

**The implementation gracefully handles non-parseable files through exception handling.** The `try/except` block catching `OSError` and `PermissionError` ensures that unreadable files (including binaries or those with restricted permissions) do not crash the incremental update, though they typically would not appear in Python source analysis candidates.

### How does hashing impact performance on large repositories?

**The overhead is negligible compared to AST parsing.** Reading file bytes and computing SHA-256 hashes occurs in milliseconds, while parsing Python source into abstract syntax trees requires orders of magnitude more processing. The time saved by skipping unchanged files far outweighs the hash computation cost, particularly in monorepos with thousands of modules.