How the Background Watcher Detects and Handles Git Changes in Codebase-Memory-MCP

The background watcher detects git changes in Codebase-Memory-MCP by polling registered repositories at adaptive intervals, comparing HEAD commits and checking working tree status via native Git commands, then triggering incremental re-indexing through a user-supplied callback.

Codebase-Memory-MCP maintains a real-time index of your repositories by running a background watcher that monitors for modifications. Unlike filesystem watchers that rely on platform-specific notifications, this implementation uses Git itself to detect changes, ensuring consistent behavior across operating systems. Understanding how the background watcher detects and handles git changes in Codebase-Memory-MCP is essential for optimizing indexing performance and resource usage.

Per-Project State Tracking

When a project is registered via cbm_watcher_watch(), the watcher allocates a project_state_t structure in src/watcher/watcher.c (lines 85-96). This struct tracks the repository's current condition:

• last_head: SHA-1 hash of the last observed Git HEAD
• is_git: Boolean flag set by verifying the path contains a .git directory via git rev-parse --git-dir
• baseline_done: Indicates whether the initial poll has completed successfully
• file_count: Approximate number of tracked files, used for adaptive timing calculations
• interval_ms: Current polling interval, calculated as base 5 seconds plus 1 second per 500 files (capped at 60 seconds)
• next_poll_ns: Absolute monotonic timestamp for the next required poll

This state persistence allows the watcher to compare historical and current repository states during each polling cycle without re-initializing Git contexts.

Baseline Initialization

The first poll for any project executes init_baseline() (lines 59-82 and 108-122 in src/watcher/watcher.c). This routine:

1. Verifies the filesystem path still exists using stat
2. Executes git -C "<path>" rev-parse --git-dir via is_git_repo() to confirm Git repository status
3. Records the current HEAD commit SHA using git_head()
4. Counts tracked files via git_file_count() (which runs git ls-files)
5. Calculates the adaptive polling interval using cbm_watcher_poll_interval_ms() and schedules the next poll by setting next_poll_ns

Once baseline initialization completes, the watcher marks baseline_done and transitions to standard change detection for subsequent polls.

Detecting Git Changes

During each polling cycle, poll_project() (lines 90-104 and 136-165) performs two Git-based checks to identify repository modifications:

HEAD Movement Detection

The watcher calls git_head(), which executes git rev-parse HEAD. If the returned SHA differs from the stored last_head, the watcher detects that a commit, checkout, pull, or merge operation has occurred since the last index.

Working Tree Status

The git_is_dirty() function runs git status --porcelain --untracked-files=normal. Non-empty output indicates modifications, staged changes, or untracked files. On non-Windows platforms, this check also inspects submodules using git submodule foreach to detect nested repository changes.

If either check reports a change, the watcher flags the project as modified and prepares to trigger re-indexing.

Triggering Incremental Re-Indexing

When changes are detected, the watcher invokes the user-provided cbm_index_fn callback with the project name and root path (lines 44-48 in src/watcher/watcher.c). This callback typically executes the full indexing pipeline for that specific project.

Following a successful re-index (return code 0), the watcher:

• Updates last_head to the current HEAD commit hash
• Refreshes file_count to adjust the adaptive polling interval
• Reschedules the next poll by setting next_poll_ns = now + interval_ms

This design ensures that indexing only occurs when Git confirms actual changes, eliminating unnecessary CPU cycles while maintaining index freshness.

Adaptive Polling and Responsive Shutdown

The watcher implements resource-conscious polling through cbm_watcher_poll_interval_ms(). The interval scales with repository size to balance responsiveness against system load, preventing excessive Git command execution on large monorepos.

To enable quick shutdown without losing change detection, cbm_watcher_run() (lines 31-44) sleeps in 500-millisecond chunks while continuously checking the stopped atomic flag. This approach allows the watcher to respond to termination requests within half a second while maintaining the adaptive polling schedule for active monitoring (lines 121-148 and 155-161).

Implementation Example

The following patterns demonstrate proper watcher setup and execution according to the src/watcher/watcher.h API:

Registering a Project and Callback

/* index_cb is called whenever a change is detected */
int index_cb(const char *project_name,
             const char *root_path,
             void *user_data) {
    /* Call the repository's indexing routine here */
    return cbm_index_project(project_name, root_path);
}

/* Create watcher (lines 53-58) */
cbm_watcher_t *watcher = cbm_watcher_new(store, index_cb, NULL);

/* Register a Git repository */
cbm_watcher_watch(watcher, "my-project", "/home/user/my-project");

Running the Background Loop

/* Run the watcher in a dedicated thread */
pthread_t th;
pthread_create(&th, NULL, (void *(*)(void *))cbm_watcher_run,
               (void *)watcher);

/* Later, request shutdown */
cbm_watcher_stop(watcher);
pthread_join(th, NULL);
cbm_watcher_free(watcher);

Manual Polling for Testing

/* Single poll execution (lines 78-109) */
int reindexed = cbm_watcher_poll_once(watcher);
printf("Projects re-indexed this cycle: %d\n", reindexed);

Summary

• The watcher maintains per-project state in project_state_t structures defined in src/watcher/watcher.c, tracking HEAD commits and repository metadata across polling cycles
• Detection relies exclusively on Git commands (git rev-parse HEAD and git status --porcelain) rather than filesystem notifications, ensuring cross-platform consistency
• Adaptive polling intervals scale from 5 seconds to 60 seconds based on tracked file counts to optimize resource usage
• Changes trigger a user-supplied callback (cbm_index_fn) that executes the indexing pipeline only when necessary
• The implementation resides primarily in src/watcher/watcher.c with public API definitions in src/watcher/watcher.h and comprehensive tests in tests/test_watcher.c

Frequently Asked Questions

How does the watcher distinguish between different types of Git operations?

The watcher detects HEAD movement via git rev-parse HEAD to identify commits, merges, or branch switches, and checks working tree dirtiness via git status --porcelain to find uncommitted changes. It treats both conditions equally as triggers for re-indexing in poll_project(), without distinguishing between specific Git operations in the detection logic.

Why does Codebase-Memory-MCP use polling instead of filesystem watchers?

The implementation uses Git-native polling because it provides consistent cross-platform behavior and detects logical changes rather than just file modifications. Filesystem watchers may miss Git-specific operations like rebases or branch switches, whereas checking HEAD commits and porcelain status ensures the indexer always reflects the actual repository state as understood by Git.

What happens if the Git repository is temporarily unavailable during a poll?

If stat fails or git rev-parse --git-dir returns an error during init_baseline() or subsequent polls, the watcher skips that project for the current cycle. The project remains registered and the watcher retries on the next scheduled poll interval, preventing transient filesystem issues from crashing the monitoring service.

How can I adjust the polling frequency for large repositories?

The polling interval automatically adapts based on file count (base 5 seconds plus 1 second per 500 files) as implemented in cbm_watcher_poll_interval_ms(), but the maximum interval caps at 60 seconds to ensure changes are detected within a reasonable timeframe. For custom behavior, you would modify the interval calculation logic in src/watcher/watcher.c before compilation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}