How the Background Auto-Sync Watcher Detects and Processes Git Changes

The background watcher implements a git-centric polling strategy that compares stored HEAD hashes and inspects git status --porcelain to trigger re-indexing callbacks whenever commits, checkouts, or working tree modifications occur.

The codebase-memory-mcp repository provides a background watcher component that continuously monitors Git repositories for changes. This C-based implementation uses an adaptive polling mechanism to detect HEAD hash mismatches and dirty working trees, ensuring the in-memory index stays synchronized with the actual codebase state.

Per-Project State Management

When a project is added via cbm_watcher_watch(), the watcher allocates a project_state_t structure (defined in src/watcher/watcher.c at lines 24–33) to track repository metadata. This structure records the project name, root path, the last known HEAD hash, a boolean flag indicating whether the path is a Git repository, an adaptive poll interval, and the next-poll deadline (next_poll_ns).

The Polling Loop and Detection Mechanism

The core polling logic resides in cbm_watcher_poll_once(), which iterates over all registered projects while respecting each project's adaptive interval computed by cbm_watcher_poll_interval_ms() (see src/watcher/watcher.c lines 82–88). For every project due for inspection, the watcher executes a two-phase detection process.

Establishing the Baseline

During the initial poll, init_baseline() validates the repository state by calling is_git_repo() (src/watcher/watcher.c lines 102–115) to verify Git initialization. It then captures the current commit hash via git_head() (lines 92–135) and counts tracked files using git_file_count() (lines 98–120), storing these values as the reference state for future comparisons.

Detecting Modifications

On subsequent polls, check_changes() (src/watcher/watcher.c lines 86–102) performs the actual change detection. The function re-reads the current HEAD hash through git_head(); a mismatch with the stored hash signals a commit, checkout, or pull operation. Additionally, git_is_dirty() executes git status --porcelain to identify uncommitted modifications, untracked files, or changes within sub-modules.

Triggering Synchronization

When either the HEAD hash or dirty status changes, the watcher invokes the user-provided index callback (index_fn) at src/watcher/watcher.c lines 45–58. This callback receives the project name, root path, and user data pointer, allowing the application to re-index the project. After successful re-indexing, the watcher refreshes the stored HEAD hash and file count, then recalculates the adaptive polling interval.

Adaptive Polling Strategy

The watcher implements an intelligent backoff mechanism through cbm_watcher_poll_interval_ms() (lines 82–88), which adjusts the polling frequency based on the project's file count. Larger repositories receive proportionally longer intervals to reduce CPU overhead, while the default base interval remains 5 seconds. This adaptive approach balances responsiveness with system resource conservation.

Run Loop and Lifecycle Management

The cbm_watcher_run() function drives the periodic polling loop, checking an atomic stopped flag to determine when to terminate. To ensure responsive shutdown, the loop sleeps in SLEEP_CHUNK_MS intervals rather than blocking for the full duration, allowing cbm_watcher_stop() to signal immediate termination (see src/watcher/watcher.c lines 21–46).

Implementation Example

The following integration demonstrates registering a callback and starting the watcher:

/* 1. Define the index callback that will be run on a change */
static int reindex_project(const char *project_name,
                           const char *root_path,
                           void *user_data)
{
    /* user_data could hold a reference to the store or any context */
    printf("Re‑indexing %s (%s)\n", project_name, root_path);
    /* …perform the expensive indexing work here… */
    return 0;   // success
}

/* 2. Create the watcher and start watching a repository */
cbm_store_t *store = cbm_store_new();                // (store creation omitted)
cbm_watcher_t *watcher = cbm_watcher_new(store,
                                         reindex_project,
                                         NULL);

/* Add a project to watch – the path must be a valid Git repo */
cbm_watcher_watch(watcher, "my‑project", "/home/user/my-project");

/* 3. Run the watcher in a background thread (or call directly) */
int rc = cbm_watcher_run(watcher, 0);   // 0 → use default base interval
if (rc != 0) {
    fprintf(stderr, "Watcher failed to start\n");
}

/* 4. When the application is shutting down */
cbm_watcher_stop(watcher);   // request a clean shutdown
cbm_watcher_free(watcher);

Summary

• The watcher maintains per-project state in project_state_t structures allocated during cbm_watcher_watch(), tracking HEAD hashes and repository metadata in src/watcher/watcher.c (lines 24–33).
• Change detection relies on comparing stored HEAD hashes and executing git status --porcelain via git_is_dirty() to identify dirty working trees.
• An adaptive polling interval calculated by cbm_watcher_poll_interval_ms() adjusts frequency based on repository size, defaulting to 5-second intervals.
• The run loop uses an atomic stopped flag and chunked sleeping to enable immediate shutdown via cbm_watcher_stop().

Frequently Asked Questions

How does the watcher distinguish between different Git operations like commits, pull, or checkout?

The watcher detects all ref-changing operations through HEAD hash comparisons in check_changes() without distinguishing the specific operation type. Whether the change results from a commit, checkout, or pull, the hash mismatch triggers the same re-indexing callback, ensuring the in-memory state reflects the current tree regardless of how it was modified.

What is the default polling interval and how does the adaptive mechanism work?

The default base polling interval is 5 seconds, passed to cbm_watcher_run(). The adaptive mechanism in cbm_watcher_poll_interval_ms() (lines 82–88) scales this interval proportionally based on the repository's tracked file count, reducing poll frequency for larger projects to minimize system load while maintaining responsiveness for smaller repositories.

How does the watcher handle shutdown while a poll is in progress?

The watcher implements cooperative cancellation through an atomic stopped flag checked in cbm_watcher_run(). Rather than blocking for the full interval, the run loop sleeps in SLEEP_CHUNK_MS chunks, allowing cbm_watcher_stop() to set the flag and trigger immediate termination even during active polling or between project checks.

Which Git commands does the watcher execute to detect changes?

The watcher executes git rev-parse HEAD (via git_head()) to retrieve the current commit hash and git status --porcelain (via git_is_dirty()) to detect working tree modifications, untracked files, and submodule changes. These commands are invoked in check_changes() at src/watcher/watcher.c lines 86–102.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}