How Incremental Analysis Works in Egonex: Re‑Scan vs. Partial Update Explained

Incremental analysis in Egonex detects changed files via Git diff, removes stale nodes and edges from the knowledge graph, and merges fresh AST data only for modified files, falling back to a full re‑scan when the change set exceeds a threshold or no previous graph exists.

The Egonex‑AI/Understand‑Anything repository implements an intelligent caching system that avoids redundant codebase analysis. By leveraging incremental analysis, the tool updates its internal knowledge graph only for files that have actually changed since the last run, significantly reducing compute time for large projects.

How Staleness Detection Drives Incremental Analysis

Before performing any update, the system must determine whether the persisted graph reflects the current state of the repository.

The isStale Helper and Git Diff Comparison

In packages/core/src/staleness.ts, the isStale function compares the current Git HEAD with the commit hash stored in the existing knowledge graph. It executes git diff to enumerate changed files since the last analysis.

// packages/core/src/staleness.ts
export function isStale(projectDir: string, lastCommitHash: string): StalenessResult {
  const changedFiles = getChangedFiles(projectDir, lastCommitHash);
  return { stale: changedFiles.length > 0, changedFiles };
}

This function returns a StalenessResult containing a boolean stale flag and an array of changedFiles. If the array is empty, the graph is current and no work is performed.

Full Re‑Scan vs. Partial Update: The Decision Logic

The orchestration layer in src/context-builder.ts evaluates the staleness result to determine whether to perform a full re‑scan or a partial update.

When Egonex Triggers a Full Re‑Scan

A full re‑scan occurs when the graph does not exist, the repository has undergone a large refactor, or the number of changed files exceeds a configurable THRESHOLD. This process walks every file in the project, extracts AST nodes via Tree‑Sitter plugins, and constructs a brand‑new knowledge graph using GraphBuilder.

Executing a Partial Update with mergeGraphUpdate

When the change set is small, the system invokes mergeGraphUpdate from packages/core/src/staleness.ts. This function surgically updates the existing graph by removing nodes associated with changed files, pruning orphaned edges, and appending freshly analyzed nodes.

// packages/core/src/staleness.ts
export function mergeGraphUpdate(
  existingGraph: KnowledgeGraph,
  changedFilePaths: string[],
  newNodes: GraphNode[],
  newEdges: GraphEdge[],
  newCommitHash: string,
): KnowledgeGraph {
  const changedSet = new Set(changedFilePaths);
  const removedNodeIds = new Set(
    existingGraph.nodes
      .filter(node => node.filePath && changedSet.has(node.filePath))
      .map(node => node.id),
  );

  const retainedNodes = existingGraph.nodes.filter(node => !removedNodeIds.has(node.id));
  const retainedEdges = existingGraph.edges.filter(
    edge => !removedNodeIds.has(edge.source) && !removedNodeIds.has(edge.target),
  );

  return {
    ...existingGraph,
    project: {
      ...existingGraph.project,
      gitCommitHash: newCommitHash,
      analyzedAt: new Date().toISOString(),
    },
    nodes: [...retainedNodes, ...newNodes],
    edges: [...retainedEdges, ...newEdges],
  };
}

Building and Merging Graph Nodes

The actual construction of AST nodes and edges happens in the analysis layer, while the merge layer ensures graph integrity.

GraphBuilder and AST Extraction

For each changed file, the GraphBuilder class in packages/core/src/analyzer/graph-builder.ts parses the source code and emits nodes representing functions, classes, and imports. The addFileWithAnalysis method attaches these nodes to a partial graph.

// packages/core/src/analyzer/graph-builder.ts
const builder = new GraphBuilder(projectName, newCommitHash);
builder.addFileWithAnalysis(filePath, analysis, meta);   // for each changed file
const partialGraph = builder.build();

The Merge Algorithm Step-by-Step

The merge algorithm guarantees consistency by performing four distinct operations:

1. Identify all nodes whose filePath property matches any entry in changedFilePaths.
2. Remove those nodes and collect their IDs into removedNodeIds.
3. Filter the edge list to exclude any edge where the source or target ID exists in removedNodeIds.
4. Append the new nodes and edges from the partial analysis and update project metadata.

This approach ensures that references to deleted or modified files are purged before the new structural data is integrated.

Orchestrating the Decision in context-builder.ts

The high‑level driver in src/context-builder.ts (re‑exported via src/index.ts) implements the decision tree that chooses between analysis modes. It checks staleness, compares changedFiles.length against THRESHOLD, and either triggers incremental processing or falls back to a complete rebuild.

// src/context-builder.ts (simplified)
const { stale, changedFiles } = isStale(projectDir, knownHash);
if (!stale) {
  // No work required – the graph is already current.
} else if (changedFiles.length < THRESHOLD) {
  // Incremental path
  const newPartial = analyzeFiles(changedFiles);
  const updatedGraph = mergeGraphUpdate(oldGraph, changedFiles, newPartial.nodes, newPartial.edges, newHash);
  saveGraph(updatedGraph);
} else {
  // Too many changes – fall back to a full scan.
  const freshGraph = runFullAnalysis(projectDir);
  saveGraph(freshGraph);
}

Summary

• Staleness detection relies on Git commit comparison via isStale in packages/core/src/staleness.ts.
• A full re‑scan rebuilds the entire knowledge graph from scratch when the change set is large or the graph is missing.
• Partial updates use mergeGraphUpdate to remove nodes and edges for changed files before merging new AST data.
• The GraphBuilder class constructs fresh nodes only for modified files, minimizing CPU and I/O overhead.
• The threshold logic in src/context-builder.ts automatically selects the most efficient analysis path.

Frequently Asked Questions

How does Egonex detect which files have changed?

The system invokes isStale from packages/core/src/staleness.ts, which runs git diff against the commit hash stored in the existing knowledge graph. It returns an array of file paths that differ between the stored state and the current HEAD.

What happens to edges when a node is removed during an incremental update?

During mergeGraphUpdate, the function collects IDs of all nodes being removed due to file changes. It then filters the existing edge list to exclude any edge where the source or target ID matches a removed node ID, effectively pruning orphaned relationships before inserting the new graph fragment.

When should I force a full re‑scan instead of relying on incremental analysis?

You should force a full re‑scan when the repository has undergone a massive refactoring that touches most files, when the persisted graph is corrupted or missing, or when you suspect the incremental merge has drifted from the true codebase state. The system automatically falls back to a full scan when changedFiles.length exceeds the internal THRESHOLD.

Where is the incremental update logic implemented in the source code?

The core logic resides in packages/core/src/staleness.ts for staleness detection and merging, while packages/core/src/analyzer/graph-builder.ts handles AST construction. The orchestration logic that decides between full and incremental analysis is located in src/context-builder.ts and re‑exported from src/index.ts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}