How the CodeGraph Engine Indexes Code Repositories in Memory Knowledge Service

The CodeGraph Engine indexes code repositories by persisting metadata to a knowledge store, serializing build jobs through a dedicated queue, and executing a worker that clones the repository and constructs a queryable dependency graph before transitioning the asset to a ready state.

The CodeGraph Engine is a core component of the Memory Knowledge Service within the TencentCloud/TencentDB-Agent-Memory repository. It treats code repositories as first-class knowledge assets, automatically orchestrating the cloning, parsing, and indexing of source code into structured graph representations. Understanding this indexing pipeline is essential for developers integrating repository intelligence into AI agents.

The Indexing Pipeline

The engine orchestrates repository ingestion through a coordinated sequence involving the CodeGraphService, a BuildQueue, and a pluggable worker function.

Creating the Metadata Record

Indexing initiates when CodeGraphService.create() is invoked. This method queries the IKnowledgeStore for an existing asset and, if absent, inserts a new row with pending status while logging an audit entry to track the creation event.

The asset initialization logic resides in [MemoryKnowledge/src/store/code-graph-service.ts](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/feat/server_team/MemoryKnowledge/src/store/code-graph-service.ts#L32-L38).

Enqueuing the Build Job

Once the record exists, enqueueBuild() pushes a CodeGraphBuildContext onto the BuildQueue. This FIFO queue guarantees that only one worker processes a given code_graph_id at any time, preventing race conditions during repository cloning.

This serialization mechanism is implemented in enqueueBuild().

Repository Cloning and Graph Construction

The real code-graph worker—defined in [MemoryKnowledge/src/module.ts](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/feat/server_team/MemoryKnowledge/src/module.ts#L119-L133)—executes the actual indexing. The worker receives the repository URL, target branch, and a temporary working directory. It performs the following operations:

• Clones the repository using git clone or fetches updates for existing checkouts.
• Runs the indexing tool (e.g., codegraph or clang-tools) to parse files, symbols, and dependencies into a graph structure.
• Returns a CodeGraphBuildResult containing the commit hash and build statistics.

State Transitions and Completion

While the worker runs, runBuild() manages the asset lifecycle. The status transitions from pending to processing (with internal states for "cloning" and "indexing"), and finally to ready upon success or failed on error.

Upon completion, onBuildComplete() generates a summary via generateCodeGraphSummary, updates the database row with statistics, logs a "ready" audit entry, and optionally notifies a configured TMC callback URL.

These state handlers are located at runBuild() and onBuildComplete().

Automatic Synchronization

The Auto-Sync Scheduler ensures that indexed graphs remain current with upstream repositories. This background component periodically scans all assets with ready status and enqueues new sync jobs. Each sync triggers the same worker pipeline to pull the latest commits and rebuild the index, maintaining synchronization without manual intervention.

The scheduler implementation resides in [MemoryKnowledge/src/store/auto-sync-scheduler.ts](https://github.com/TencentCloud/TencentDB-Agent-Memory/blob/feat/server_team/MemoryKnowledge/src/store/auto-sync-scheduler.ts).

Deletion and Cancellation Flow

Assets can be removed safely even during active indexing. The delete() method adds the code_graph_id to an internal cancelled set. Workers check isDeleted() at safe checkpoints and abort execution early if flagged.

After cancellation or direct deletion, cleanupResources() removes the in-memory instance, deletes the database row via IKnowledgeStore, and recursively removes the on-disk working directory using rmSync.

See the deletion logic in delete() and isDeleted().

Retrieving Indexed Graphs

Consumers access indexed data through CodeGraphService.get(), list(), or getById(). These methods return the current metadata, processing status, and a service_url pointing to the HTTP endpoint serving the graph data.

Query methods are defined at get() and list().

Programmatic Usage

The following example demonstrates initializing the service with a concrete store and worker, creating a code-graph asset, triggering a manual sync, and cleaning up:

import { CodeGraphService } from "./src/store/code-graph-service";
import { memoryStore } from "./src/store/sqlite-store";
import { realCodeWorker } from "./src/module";

// Initialize the service with dependencies
const cgService = new CodeGraphService({
  store: memoryStore,
  dataRoot: "/var/lib/memory/code-graphs",
  worker: realCodeWorker,
  logger: console,
});

// 1. Create a new code-graph asset
const { row, existed } = cgService.create({
  service_id: "codegraph",
  team_id: "team123",
  repo_url: "https://github.com/example/repo.git",
  branch: "main",
});
console.log(`Created new asset: ${!existed}, ID: ${row.code_graph_id}`);

// 2. Trigger a manual sync to pull latest changes
const syncResult = cgService.sync("codegraph", "team123", row.code_graph_id, "user42");
if (syncResult.kind === "ok") {
  console.log("Sync enqueued successfully");
}

// 3. Retrieve current status
const info = cgService.get("codegraph", "team123", row.code_graph_id);
console.log(`Current status: ${info?.status}`);

// 4. Delete the asset and cancel any running worker
cgService.delete("codegraph", "team123", row.code_graph_id);

Summary

• The CodeGraph Engine treats repositories as versioned knowledge assets managed through CodeGraphService.
• BuildQueue serializes indexing jobs to prevent concurrent mutations of the same repository.
• The realCodeWorker clones repositories and runs the graph construction tooling in isolated temporary directories.
• A state machine tracks assets from pending through processing to ready or failed, with full audit logging.
• Auto-sync periodically refreshes ready assets to incorporate new commits automatically.
• Safe cancellation and cleanup mechanisms ensure resources are released even when builds are interrupted.

Frequently Asked Questions

What triggers the CodeGraph Engine to start indexing a repository?

Indexing is triggered by calling CodeGraphService.create(), which initializes the metadata record and immediately enqueues a build job. The Auto-Sync Scheduler can also trigger re-indexing for existing assets that are already in the ready state.

How does the engine handle concurrent indexing requests for the same repository?

The BuildQueue ensures serial execution by processing only one job per code_graph_id at a time. Subsequent requests for the same asset wait in the queue until the current worker completes, preventing race conditions during file system operations.

What happens if a repository deletion is requested during active indexing?

The delete() method adds the asset ID to a cancellation set. The active worker checks isDeleted() at safe intervals and aborts execution early. The service then invokes cleanupResources() to remove the database entry and delete the cloned repository from disk.

How does the auto-sync feature keep code graphs up to date?

The Auto-Sync Scheduler periodically queries the IKnowledgeStore for all assets with ready status. For each asset, it enqueues a new build job that executes the same worker pipeline—pulling the latest commits via git fetch and rebuilding the graph—ensuring the indexed data reflects the current state of the repository.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}