How to Implement Auto-Sync for CodeGraph Repositories in TencentDB-Agent-Memory

The auto-sync feature in TencentDB-Agent-Memory uses the AutoSyncScheduler class to periodically scan for ready CodeGraph repositories and process them through a managed worker pool that invokes CodeGraphService.sync(), ensuring your semantic code index stays current without manual intervention.

The TencentDB-Agent-Memory repository provides an intelligent memory layer for database agents, with its MemoryKnowledge service handling the ingestion and synchronization of code repositories. Implementing auto-sync for CodeGraph repositories ensures that your knowledge graph reflects the latest commits and structural changes automatically. This article examines the production-ready implementation found in the open-source codebase, detailing the scheduler architecture, configuration options, and integration patterns.

Understanding the AutoSyncScheduler Architecture

The synchronization system centers on the AutoSyncScheduler class located in MemoryKnowledge/src/store/auto-sync-scheduler.ts. This component orchestrates a multi-layered pipeline that balances throughput with resource constraints.

The Scheduler Component

At the heart of the system lies a FIFO queue combined with a fixed-size worker pool. The scheduler scans for ready repositories every scanIntervalMs, deduplicating entries using an inFlight Set to prevent duplicate processing. When scan() identifies eligible CodeGraphRow entries from the store, it enqueues only those not already queued or actively syncing.

Worker Pool and Concurrency Control

Up to maxConcurrentSyncs coroutines run continuously, pulling jobs from the queue and executing cgService.sync(row). These workers are designed to swallow errors gracefully, ensuring that a single repository failure does not block the entire synchronization pipeline. When the queue empties, workers sleep for WORKER_IDLE_POLL_MS before polling again, preventing CPU churn.

Configuration via Environment Variables

The resolveAutoSyncConfig helper function parses three critical environment variables to control behavior:

• KNOWLEDGE_AUTO_SYNC_ENABLED: Boolean flag to activate the scheduler
• KNOWLEDGE_AUTO_SYNC_SCAN_INTERVAL_MIN: Defines the scanIntervalMs between automated scans
• KNOWLEDGE_AUTO_SYNC_MAX_CONCURRENT: Sets the maxConcurrentSyncs limit for the worker pool

Values are automatically clamped to safe defaults during initialization.

HTTP Admin API Endpoints

The system exposes administrative controls through MemoryKnowledge/src/routes/auto-sync.ts, mounting two routes under the /v3 prefix:

• GET /auto-sync/status: Returns real-time metrics including running, activeSyncs, queueLength, and scanning state
• POST /auto-sync/trigger: Allows manual invocation of scheduler.scan() for immediate processing

Server Integration Pattern

The scheduler integrates into the application lifecycle through createKnowledgeModule in MemoryKnowledge/src/module.ts, which instantiates the scheduler and injects it into route handlers. The createApp function in MemoryKnowledge/src/server.ts (lines 90-95) ultimately starts the service, ensuring the auto-sync system initializes alongside other knowledge store components.

Auto-Sync Workflow Explained

Understanding the execution flow helps developers debug synchronization issues and optimize performance.

Initialization and Startup Sequence

When scheduler.start() is called, the system implements a 30-second delay to allow database restoration to complete. Following this grace period, the scheduler initiates a timer for periodic scans and spawns the configured number of worker coroutines. This delayed start prevents race conditions during service startup.

Repository Scanning and Deduplication

The scan() method queries the store via store.listReadyCodeGraphs() to retrieve all repositories marked for synchronization. Before enqueuing, the system checks the inFlight Set and current queue contents, filtering out any repositories already being processed. This deduplication mechanism is critical for preventing resource contention when scans overlap with long-running syncs.

Worker Execution and Error Handling

Each worker enters a continuous loop that:

1. Pops the next CodeGraphRow from the FIFO queue
2. Invokes await this.cgService.sync(job) to fetch Git updates and rebuild the graph
3. Updates internal metrics and logging
4. Handles errors without propagating them, allowing the worker to immediately pick up the next job

This design ensures high availability even when individual repositories contain corrupted data or network issues.

Implementing Auto-Sync in Your Deployment

Deploying the auto-sync feature requires proper configuration and understanding of the administrative interfaces.

Enabling and Configuring the Scheduler

To activate automatic synchronization, set the required environment variables and instantiate the scheduler in your bootstrap code:

import { resolveAutoSyncConfig, AutoSyncScheduler } from "./store/auto-sync-scheduler.js";

const config = resolveAutoSyncConfig();
const scheduler = new AutoSyncScheduler({
  store: knowledgeStore,
  cgService: codeGraphService,
  config,
});

scheduler.start(); // Begins 30s delayed startup and worker pool

The configuration object automatically handles type coercion and boundary checking for concurrency limits.

Monitoring via Admin API Endpoints

For operational visibility, query the scheduler state using the built-in HTTP endpoints:

# Check current synchronization status

curl http://localhost:8080/v3/auto-sync/status

# Force immediate repository scan

curl -X POST http://localhost:8080/v3/auto-sync/trigger

The status endpoint returns JSON containing queueLength (pending repositories), activeSyncs (currently processing), and scanning (boolean indicating active scan).

Customizing and Extending Auto-Sync

The modular architecture allows teams to adapt the synchronization logic to specific requirements without modifying core infrastructure.

Modifying Scan Criteria

To adjust which repositories qualify for synchronization, edit the scan() method in MemoryKnowledge/src/store/auto-sync-scheduler.ts. Modify the query logic that calls store.listReadyCodeGraphs() to filter by additional metadata fields, such as repository size, last sync timestamp, or custom tags defined in CodeGraphRow interfaces.

Adjusting Worker Concurrency

Change the KNOWLEDGE_AUTO_SYNC_MAX_CONCURRENT environment variable to scale the worker pool horizontally. Alternatively, pass a custom maxConcurrentSyncs value directly to the AutoSyncScheduler constructor config object. Higher values increase throughput but consume more memory and Git operation handles.

Adding Retry Logic to Workers

While the default implementation swallows errors to maintain throughput, you can implement exponential backoff by modifying the worker loop:

// Inside runWorker (auto-sync-scheduler.ts)
while (!this.stopped) {
  const job = this.queue.shift();
  if (!job) {
    await new Promise(r => setTimeout(r, WORKER_IDLE_POLL_MS));
    continue;
  }
  try {
    await this.cgService.sync(job);
    attempts = 0; // Reset on success
  } catch (e) {
    // Exponential backoff retry
    await new Promise(r => setTimeout(r, 2 ** attempts * 1000));
    attempts++;
    if (attempts > 5) {
      console.error(`Failed to sync ${job.id} after 5 attempts`);
      attempts = 0; // Reset and move to next job
    }
  }
}

This pattern preserves queue order while providing resilience against transient network failures.

Summary

Implementing auto-sync for CodeGraph repositories in TencentDB-Agent-Memory involves configuring the AutoSyncScheduler component with environment variables and integrating it into your service bootstrap. Key implementation points include:

• The scheduler uses a FIFO queue and inFlight Set deduplication in MemoryKnowledge/src/store/auto-sync-scheduler.ts to manage repository processing
• Worker pools limited by KNOWLEDGE_AUTO_SYNC_MAX_CONCURRENT execute CodeGraphService.sync() while isolating errors to prevent cascade failures
• A mandatory 30-second startup delay ensures database restoration completes before the first scan
• Administrative visibility comes through /v3/auto-sync/status and manual trigger capabilities via /v3/auto-sync/trigger
• Configuration relies on KNOWLEDGE_AUTO_SYNC_ENABLED, KNOWLEDGE_AUTO_SYNC_SCAN_INTERVAL_MIN, and KNOWLEDGE_AUTO_SYNC_MAX_CONCURRENT environment variables

Frequently Asked Questions

How do I enable automatic synchronization for CodeGraph repositories?

Set the KNOWLEDGE_AUTO_SYNC_ENABLED environment variable to true and ensure your bootstrap code calls scheduler.start() after instantiating AutoSyncScheduler with valid store and cgService dependencies. The scheduler will begin scanning after a mandatory 30-second initialization delay designed to prevent database connection race conditions.

What happens if a repository synchronization fails?

Individual worker coroutines catch and swallow errors during cgService.sync() execution, ensuring that a single repository failure does not block the processing queue or affect other concurrent synchronizations. Failed jobs are simply logged and removed from the queue; the worker immediately picks up the next available repository without requiring manual intervention.

Can I trigger a manual synchronization scan outside the normal schedule?

Yes. The HTTP admin API exposed in MemoryKnowledge/src/routes/auto-sync.ts provides a POST /v3/auto-sync/trigger endpoint that invokes scheduler.triggerScan(). This immediately executes the scan() method without disrupting the existing periodic timer interval, allowing administrators to force updates when new code is pushed to critical repositories.

How does the system prevent duplicate synchronization of the same repository?

The AutoSyncScheduler maintains an inFlight Set tracking repository IDs currently being processed, combined with a FIFO queue that checks for existing entries before enqueuing new jobs. During each scan() operation, the system filters results from store.listReadyCodeGraphs() against both the queue contents and the inFlight Set, ensuring each repository syncs atomically without parallel duplicate workers.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}