How the Background Watcher Detects and Handles File Changes in codebase-memory-mcp

The background watcher in DeusData/codebase-memory-mcp uses a hybrid event-driven and polling architecture to monitor registered project directories, invoking re-indexing callbacks when file changes are detected while managing edge cases like temporary root unavailability through configurable grace windows and prune streak counters.

The watcher subsystem maintains synchronization between the in-memory representation of repositories and the actual file system. Implemented in C within the src/watcher/ directory, this component provides a robust abstraction over platform-specific notification APIs with a portable fallback mechanism.

Watcher Initialization and Project Registration

The watcher lifecycle begins with cbm_watcher_new(), which requires a backing store, an index callback function, and optional user data. This callback receives notifications whenever the watcher detects changes requiring re-indexing of the affected project.

Projects are registered using cbm_watcher_watch(), which accepts a project identifier and root directory path. The watcher maintains per-project state including last-known modification timestamps and a prune streak counter. Notably, registering a project that already exists replaces the old path without incrementing the watch count, as reported by cbm_watcher_watch_count().

/* Create a watcher with an index callback */
cbm_watcher_t *watcher = cbm_watcher_new(store, index_callback, NULL);

/* Register a project for monitoring */
cbm_watcher_watch(watcher, "my-project", "/home/user/my-project");

Change Detection: inotify and Polling Fallback

The detection mechanism operates in two tiers depending on platform capabilities. On Linux systems, the watcher first attempts to use inotify for event-driven notifications. When inotify is unavailable or the platform does not support it, the system falls back to a poll-based scan implemented in cbm_watcher_poll_once().

The polling mechanism walks the directory tree and compares stat() results against saved snapshots to identify new, modified, or removed files. The main event loop repeatedly calls cbm_watcher_poll_once() (or lets the inotify thread dispatch events), triggering the callback when discrepancies are found.

/* Main loop – poll for changes */
while (running) {
    int reindexed = cbm_watcher_poll_once(watcher);
    if (reindexed) {
        printf("Project was re-indexed due to file changes.\n");
    }
    sleep(1);
}

Handling Edge Cases and Grace Windows

Beyond standard change detection, the implementation handles several edge cases through specific mechanisms:

• Missing Root Detection: The cbm_watcher_root_missing_errno() function classifies ENOENT and ENOTDIR errors as "root missing" conditions, distinguishing between temporary and permanent project unavailability.

• Grace Window Management: The watcher implements a configurable grace period to prevent immediate pruning of projects that disappear temporarily. The prune streak increments only after this window expires, allowing for transient file system events without triggering cleanup.

• Touch Mechanism: Calling cbm_watcher_touch() with a project identifier resets the prune streak, explicitly signaling that the project remains active even if the root directory experienced temporary unavailability.

/* If the project root is temporarily unavailable, keep it alive */
cbm_watcher_touch(watcher, "my-project");

Configuration Limits and Safety Guarantees

The polling frequency is adjustable via cbm_watcher_poll_interval_ms(), which enforces strict bounds: the interval clamps to a minimum of 500 ms and a maximum of 100,000 ms (100 seconds) regardless of input values.

All public functions in the watcher API tolerate NULL arguments safely. These calls become no-operations and return safe defaults rather than causing undefined behavior, ensuring stability during long-running monitoring operations.

Source Code Organization

The watcher functionality spans three primary artifacts within the repository:

• src/watcher/watcher.h declares the public API including all cbm_watcher_* functions and type definitions
• src/watcher/watcher.c contains the core implementation covering registration, the hybrid inotify/polling logic, and pruning decisions
• tests/test_watcher.c provides exhaustive unit tests documenting poll interval limits, grace-window behavior, and null-safety guarantees

Summary

• The background watcher uses a hybrid inotify (Linux) and poll-based scanning approach to detect file changes
• Registration occurs through cbm_watcher_new() and cbm_watcher_watch(), with per-project state tracking including modification timestamps
• Grace windows and prune streaks managed by cbm_watcher_touch() handle temporary root directory unavailability
• Poll intervals are configurable between 500 ms and 100,000 ms via cbm_watcher_poll_interval_ms()
• All public API functions are null-safe, returning safe defaults rather than crashing on invalid inputs
• Implementation resides in src/watcher/watcher.c and src/watcher/watcher.h

Frequently Asked Questions

How does the watcher decide between inotify and polling?

The watcher attempts to use inotify first on Linux systems for event-driven efficiency. If inotify is unavailable or the platform does not support it, the implementation automatically falls back to the poll-based scan in cbm_watcher_poll_once(). This scan walks the directory tree and compares current stat() timestamps against cached values to detect modifications.

What happens when a project root directory temporarily disappears?

The watcher implements a grace window mechanism to handle transient unavailability. When cbm_watcher_root_missing_errno() detects ENOENT or ENOTDIR errors, it increments a prune streak only after the configurable grace period expires. Calling cbm_watcher_touch() resets this counter, allowing applications to signal that a temporarily missing project should not be pruned from the watch list.

How is the polling interval configured and what are its limits?

Applications adjust the polling frequency using cbm_watcher_poll_interval_ms(). The implementation enforces bounds checking that clamps any input to a minimum of 500 milliseconds and a maximum of 100,000 milliseconds, ensuring the watcher cannot be configured to poll excessively fast or slow.

Is the watcher thread-safe for concurrent access?

The source code analysis indicates that the watcher is designed to run once at application startup with the main event loop calling cbm_watcher_poll_once(). While the API is null-safe, the implementation suggests that for inotify-based operations, a separate thread may dispatch events, though explicit synchronization mechanisms are not detailed in the public header interface.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}