# How Auto-Index and Git-Watcher Detect File Changes in Codebase-Memory-MCP

> Discover how Auto-Index and Git-Watcher detect file changes in Codebase-Memory-MCP. Learn about background scans, continuous polling, and FNV-1a signature triggers for efficient code updates.

- Repository: [Martin Vogel/codebase-memory-mcp](https://github.com/DeusData/codebase-memory-mcp)
- Tags: how-to-guide
- Published: 2026-07-28

---

**The DeusData/codebase-memory-mcp repository uses a dual-stage detection system where auto-index performs a one-time background scan at session startup, while the Git-watcher continuously polls via `git rev-parse HEAD` and `git status --porcelain` to detect commit changes and dirty worktree states, triggering re-indexing whenever the FNV-1a signature of file metadata changes.**

Codebase-Memory-MCP implements an efficient file change detection architecture that avoids costly filesystem scans by leveraging Git plumbing commands. Both the **auto-index** and **git-watcher** features rely on Git-native tooling—specifically `git rev-parse`, `git status`, and `git ls-files`—to monitor repository state with minimal overhead. The system divides responsibilities between a one-shot initialization thread and a long-running polling daemon, coordinating through shared pipeline functions in [`src/mcp/mcp.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/mcp/mcp.c) and [`src/watcher/watcher.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/watcher/watcher.c).

## Auto-Index: One-Shot Detection at Session Startup

The auto-index feature triggers a background indexing run when a new session starts, but only if no existing database is detected. This mechanism executes in a separate thread and serves as the initial bootstrap before continuous monitoring begins.

### Session Initialization and Database Check

When the server creates a session in [`src/mcp/mcp.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/mcp/mcp.c), the `maybe_auto_index()` function checks for an existing `.db` file using `cbm_file_size(db_check)`. If a database already exists, the system skips indexing and immediately registers the project with the Git-watcher via `register_watcher_if_enabled()`.

### Configuration and File Count Guards

Before launching the indexing thread, the system validates two configuration parameters via `cbm_config_get_bool()` and `cbm_config_get_int()`:

- **auto_index**: Boolean flag defaulting to *off*
- **auto_index_limit**: Integer defaulting to `50000` files

The function `cbm_mcp_auto_index_within_file_limit()` executes `git ls-files -z` with a 5-second timeout to count tracked files. Projects exceeding the limit abort automatically to prevent resource exhaustion.

### Background Thread Execution

If all guards pass, `cbm_thread_create()` launches `autoindex_thread`, which instantiates a full-mode pipeline via `cbm_pipeline_new()` and executes `cbm_pipeline_run()`. Upon successful completion, the thread calls `register_watcher_if_enabled()` to transition the project to continuous monitoring mode.

## Git-Watcher: Continuous Change Detection

While auto-index handles initial ingestion, the Git-watcher provides ongoing surveillance through an adaptive polling mechanism defined in [`src/watcher/watcher.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/watcher/watcher.c).

### Project Registration and Baseline Establishment

The `cbm_watcher_watch()` function stores a `project_state_t` entry for each monitored project. During the first poll cycle, `init_baseline()` executes three Git commands to establish the reference state:

1. `git rev-parse --git-dir` to verify repository validity
2. `git rev-parse HEAD` to capture the current commit hash
3. `git ls-files -z` to enumerate tracked files

### Adaptive Polling Intervals

Rather than using a fixed polling rate, the watcher calculates dynamic intervals via `cbm_watcher_poll_interval_ms()`. The formula adds 1 second per 500 tracked files to a base of 5 seconds, capping at 60 seconds. This prevents excessive CPU usage in large monorepos while maintaining responsiveness in smaller projects.

### Dual-Strategy Change Detection

On each poll, `check_changes()` evaluates two independent signals:

**HEAD Movement Detection**: The watcher compares the output of `git rev-parse HEAD` against the stored `last_head` value. Any divergence—caused by new commits, checkouts, or pulls—triggers a re-index.

**Dirty-State Signature Calculation**: For uncommitted changes, the system executes `git status --porcelain -uall -z` and computes an FNV-1a hash using `sig_fold_path_stat` macros that incorporate file paths, sizes, and modification times. If the resulting signature differs from `last_dirty_sig`, the worktree is considered modified.

### Re-Index Triggering and State Commitment

When either detection method identifies changes, `poll_project()` invokes the server-provided `index_fn` callback—the same function used by auto-index. Only after successful completion (`rc == 0`) does the watcher commit the new HEAD hash and dirty signature to the baseline, ensuring atomic state transitions.

### Stale Project Pruning

The watcher includes garbage collection via `prune_missing_project()`. If a project's root directory disappears for a configurable number of polls (`MISSING_ROOT_DELETE_AFTER`) and exceeds the grace period (`PRUNE_GRACE_DEFAULT_S`), the system calls `delete_cached_project_db()` to remove the cached database and deletes the entry from the watch list.

## Configuration and Integration Examples

### Enabling Auto-Index via CLI

Configure the system to automatically index projects on session startup:

```bash

# Enable auto-index globally (disabled by default)

codebase-memory-mcp config set auto_index true

# Set maximum file count threshold (default: 50000)

codebase-memory-mcp config set auto_index_limit 20000

# Start session to trigger background indexing

codebase-memory-mcp session start /path/to/project

```

When the session initializes, `maybe_auto_index()` in [`src/mcp/mcp.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/mcp/mcp.c) launches the background thread. Upon completion, the project automatically registers with the Git-watcher for continuous monitoring.

### Programmatic Watcher Registration

Integrate the watcher directly using the C API:

```c
// Register a project for continuous monitoring
bool success = cbm_watcher_watch(
    srv->watcher,
    "project-name",           // Unique project identifier
    "/absolute/path/to/repo"  // Repository root
);

if (!success) {
    fprintf(stderr, "Watcher registration failed\n");
}

```

This call creates a `project_state_t` entry in the watcher's hash table, beginning the adaptive polling cycle.

### Custom Index Callback Implementation

Define a custom indexing function to handle detected changes:

```c
int custom_indexer(const char *project, const char *root, void *userdata)
{
    // Execute full indexing pipeline
    cbm_pipeline_t *pipeline = cbm_pipeline_new(root, NULL, CBM_MODE_FULL);
    int result = cbm_pipeline_run(pipeline);
    return result; // Return 0 on success
}

// Initialize watcher with custom callback
cbm_watcher_t *watcher = cbm_watcher_new(store, custom_indexer, NULL);

```

The watcher invokes this callback whenever `check_changes()` detects HEAD movement or dirty-state modifications.

## Summary

- **Auto-index** performs a one-time background scan at session startup using `git ls-files` to count and index tracked files, running in a separate thread spawned by `cbm_thread_create()` in [`src/mcp/mcp.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/mcp/mcp.c).
- **Git-watcher** continuously monitors repositories through adaptive polling in [`src/watcher/watcher.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/watcher/watcher.c), calculating intervals based on repository size (5s base + 1s per 500 files, capped at 60s).
- **Change detection** relies on two Git-native signals: HEAD hash comparison via `git rev-parse HEAD` and dirty-state signatures computed via `git status --porcelain` with FNV-1a hashing.
- **Re-indexing** occurs only when the watcher detects state changes, invoking the same pipeline used by auto-index through the configurable `index_fn` callback.
- **Resource protection** includes file-count limits, configurable timeouts, and automatic pruning of missing projects via `prune_missing_project()` after grace periods.

## Frequently Asked Questions

### How does the Git-watcher detect uncommitted changes?

The Git-watcher detects uncommitted changes by executing `git status --porcelain -uall -z` and computing an FNV-1a hash of the output using `sig_fold_path_stat`. This hash incorporates file paths, sizes, and modification times into a `last_dirty_sig` value. When the computed signature differs from the stored baseline, the system recognizes the worktree as dirty and triggers re-indexing through the `index_fn` callback.

### What happens if a project has too many files for auto-indexing?

If `cbm_mcp_auto_index_within_file_limit()` determines the file count exceeds the `auto_index_limit` configuration (defaulting to 50000 files) via `git ls-files -z`, the auto-index aborts before launching the background thread. The system still registers the project with the Git-watcher if enabled, allowing manual or on-demand indexing while preventing resource exhaustion during automatic scans.

### Can the polling interval be customized?

The polling interval adapts automatically based on repository size through `cbm_watcher_poll_interval_ms()` in [`src/watcher/watcher.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/watcher/watcher.c). While not directly configurable via user settings, the algorithm uses a base of 5 seconds plus 1 second per 500 tracked files, capped at 60 seconds. This ensures large monorepos don't overwhelm system resources while smaller projects receive frequent checks.

### How does the system handle deleted or moved project directories?

The Git-watcher implements stale-root pruning via `prune_missing_project()` in [`src/watcher/watcher.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/watcher/watcher.c). If `git rev-parse --git-dir` fails or the directory disappears for more than `MISSING_ROOT_DELETE_AFTER` polls (with a grace period of `PRUNE_GRACE_DEFAULT_S`), the watcher automatically calls `delete_cached_project_db()` to remove the cached database and deletes the project entry from the watch list.