How the Background Watcher Detects Git Changes in Codebase Memory MCP

The background watcher detects git changes by polling registered projects at adaptive intervals, comparing stored HEAD hashes against current repository state, and executing git status --porcelain to identify dirty working trees, triggering a re-index callback whenever discrepancies are found.

The Codebase Memory MCP server maintains real-time synchronization between its in-memory index and actual Git repositories through a dedicated background watcher thread. This component continuously monitors project directories for commits, checkouts, pulls, and uncommitted modifications. Understanding how the background watcher detects git changes reveals the system's efficient, git-centric polling strategy implemented in src/watcher/watcher.c.

Per-Project State Tracking

When a project is registered via cbm_watcher_watch(), the watcher allocates a project_state_t structure (see src/watcher/watcher.c, lines 24-33). This structure maintains the project name, root path, last known HEAD hash, a flag indicating whether the path is a Git repository, an adaptive poll interval, and the next poll deadline (next_poll_ns).

Adaptive Polling and Change Detection

The core detection logic resides in cbm_watcher_poll_once(), which iterates through all registered projects while respecting each project's adaptive interval computed by cbm_watcher_poll_interval_ms() (lines 82-88).

Baseline Initialization

For new projects, init_baseline() establishes the initial state by:

• Verifying Git repository status via is_git_repo() (lines 102-115)
• Capturing the current HEAD commit via git_head() (lines 92-135)
• Counting tracked files via git_file_count() (lines 98-120) to determine the poll frequency

Detecting Repository Changes

On subsequent polls, check_changes() (lines 86-102) performs dual validation:

1. HEAD Hash Comparison: Re-reads the current HEAD via git_head(). A mismatch with the stored hash signals a commit, checkout, or pull operation.

2. Dirty Working Tree Detection: Executes git status --porcelain through git_is_dirty() to identify uncommitted modifications, untracked files, or submodule changes.

When either check returns positive, the watcher invokes the user-provided index_fn callback (see call site at lines 45-58) to re-index the project. After successful re-indexing, the stored HEAD and file count refresh, and a new adaptive interval calculates.

Run Loop and Shutdown Mechanics

The cbm_watcher_run() function drives the periodic polling with a default base interval of 5 seconds. The loop checks an atomic stopped flag and sleeps in short chunks using SLEEP_CHUNK_MS to enable responsive termination when cbm_watcher_stop() is called (lines 21-30, 31-46).

Implementation Example

/* 1. Define the index callback that will be run on a change */
static int reindex_project(const char *project_name,
                           const char *root_path,
                           void *user_data)
{
    /* user_data could hold a reference to the store or any context */
    printf("Re-indexing %s (%s)\n", project_name, root_path);
    /* …perform the expensive indexing work here… */
    return 0;   // success
}

/* 2. Create the watcher and start watching a repository */
cbm_store_t *store = cbm_store_new();                // (store creation omitted)
cbm_watcher_t *watcher = cbm_watcher_new(store,
                                         reindex_project,
                                         NULL);

/* Add a project to watch – the path must be a valid Git repo */
cbm_watcher_watch(watcher, "my-project", "/home/user/my-project");

/* 3. Run the watcher in a background thread (or call directly) */
int rc = cbm_watcher_run(watcher, 0);   // 0 → use default base interval
if (rc != 0) {
    fprintf(stderr, "Watcher failed to start\n");
}

/* 4. When the application is shutting down */
cbm_watcher_stop(watcher);   // request a clean shutdown
cbm_watcher_free(watcher);

Summary

• The watcher maintains per-project state in project_state_t structures tracking HEAD hashes, poll intervals, and next deadlines
• Detection occurs through check_changes(), which compares stored HEAD hashes against current repository state and checks dirty status via git status --porcelain
• Polling frequency adapts based on file count via cbm_watcher_poll_interval_ms() to optimize system resources
• Changes trigger the index_fn callback to synchronize the in-memory index with the repository state
• The run loop uses chunked sleeping (SLEEP_CHUNK_MS) for responsive shutdown handling via cbm_watcher_stop()

Frequently Asked Questions

How does the watcher identify if a project path is a Git repository?

The is_git_repo() function in src/watcher/watcher.c (lines 102-115) validates the project path before establishing a baseline. This ensures only valid Git repositories enter the monitoring cycle, preventing errors on non-Git directories.

What conditions trigger a re-index operation?

A re-index triggers when check_changes() detects either a HEAD hash mismatch (indicating commits, checkouts, or pulls) or a dirty working tree (uncommitted changes detected via git status --porcelain). Either condition causes the watcher to invoke the registered index_fn callback to update the in-memory representation.

How does the adaptive polling interval work?

The cbm_watcher_poll_interval_ms() function calculates intervals based on the project's tracked file count recorded during git_file_count(). Larger repositories receive longer intervals between checks to reduce system load, while smaller projects poll more frequently for faster change detection.

Can the watcher detect changes in real-time?

The implementation uses polling rather than filesystem events, with a default base interval of 5 seconds. While not truly real-time, the chunked sleep mechanism in cbm_watcher_run() allows for sub-second response times to shutdown requests while maintaining regular polling intervals for change detection.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}