How Auto-Index Works with Git-Based Change Detection in codebase-memory-mcp

The codebase-memory-mcp server automatically indexes projects on startup and monitors both the working tree and .git directory for changes, triggering re-indexes via filesystem polling to keep the code graph synchronized with the repository state.

The codebase-memory-mcp repository provides a Model Context Protocol (MCP) server that maintains a searchable graph of your codebase. Understanding how auto-index works with git-based change detection in codebase-memory-mcp enables real-time code intelligence without manual re-indexing, as the system detects Git operations—commits, branch switches, and resets—through filesystem monitoring.

The Three-Stage Auto-Index Initialization Process

When the server starts, it executes a coordinated three-stage pipeline to establish the session and begin monitoring for changes.

Stage 1: Detecting the Session Root

The process begins with detect_session() in src/mcp/mcp.c (lines 54–78). This function examines the current working directory (CWD) to determine the session root. If the CWD is neither the root filesystem (/) nor the user’s home directory, it becomes the project root. The function then generates a project identifier using cbm_project_name_from_path, which creates a unique name based on the directory path.

Stage 2: Launching the Background Thread

Once the session root is established, maybe_auto_index() (defined in src/mcp/mcp.c, lines 59–97 and 110–132) determines whether indexing is required. This function checks for the existence of a <project>.db file in the cache directory. If the database is missing and the auto_index configuration flag is enabled, it spawns an autoindex_thread.

The background thread first attempts a supervised subprocess via index_run_supervised_path. If this external process fails, it falls back to an in-process pipeline using cbm_pipeline_new followed by cbm_pipeline_run. After the indexing completes, the thread calls register_watcher_if_enabled() to activate change detection.

Stage 3: Registering the Git Change Watcher

The final initialization step occurs in register_watcher_if_enabled() (src/mcp/mcp.c, lines 96–103). This function registers the project with the global watcher by invoking cbm_watcher_watch, which begins polling the filesystem—including the .git directory—for modifications. This registration can be disabled by setting the auto_watch configuration flag to false.

How Git Changes Trigger Re-Indexing

The watcher does not parse Git internals or hook into Git events directly. Instead, it treats the .git directory as part of the standard filesystem watch list. When cbm_watcher_poll_once (implemented in src/watcher/watcher.c) detects any filesystem activity in the working tree or the .git folder—such as new commits, branch switches, or hard resets—it invokes the index callback supplied during server initialization.

This callback triggers the same indexing pipeline used during the initial auto-index, ensuring the stored code graph remains consistent with the repository’s current state without requiring manual intervention.

Configuration Flags

Three configuration flags control the auto-index and git-based change detection behavior:

• auto_index – Enables the background auto-index thread when a new session starts. Default: false.
• auto_index_limit – Sets the maximum number of files that may be indexed automatically. Default: 50000.
• auto_watch – Determines whether the filesystem watcher is registered after indexing completes. Default: true.

Implementation Code Examples

Enable auto-indexing and adjust the file limit via the CLI:

# Enable auto-index for new sessions

codebase-memory-mcp config set auto_index true

# Raise the file-count limit for large repositories

codebase-memory-mcp config set auto_index_limit 100000

When the server starts (e.g., codebase-memory-mcp serve), the initialization logic in src/main.c wires the components together:

/* In src/main.c – server initialization */
if (srv->config) {
    // Configuration loaded from CLI and config files
}
if (srv->session_root[0]) {
    maybe_auto_index(srv);          // Launches background autoindex_thread
}

While the server runs, the watcher polls for Git activity. A simplified view of the detection logic in src/watcher/watcher.c shows the callback mechanism:

/* In watcher.c – filesystem polling loop */
void cbm_watcher_poll_once(cbm_watcher_t *w) {
    if (filesystem_changed(w->rootdir) || git_dir_changed(w->rootdir)) {
        /* Trigger the index callback supplied by the server */
        w->index_callback(w->project);
    }
}

Summary

• detect_session() in src/mcp/mcp.c identifies the project root and generates a unique project name from the path.
• maybe_auto_index() launches a background thread that indexes the project via supervised subprocess or in-process pipeline if the database does not exist.
• register_watcher_if_enabled() activates a filesystem watcher that polls the working tree and .git directory for changes.
• Changes detected by the watcher trigger re-indexing through the same pipeline, ensuring the code graph stays synchronized with Git operations.
• Configuration flags auto_index, auto_index_limit, and auto_watch control behavior through the CLI interface defined in src/cli/cli.c.

Frequently Asked Questions

Does the watcher parse Git commit history or hooks to detect changes?

No. According to the source code in src/watcher/watcher.c, the watcher performs filesystem polling on the working tree and the .git directory. It detects changes through standard filesystem modification events rather than parsing Git internals or using Git hooks, making it agnostic to how Git operations are performed.

What happens if the initial supervised indexing process fails?

If index_run_supervised_path fails in the autoindex_thread, the system falls back to an in-process pipeline using cbm_pipeline_new and cbm_pipeline_run as implemented in src/mcp/mcp.c (lines 110–132). This ensures indexing completes even when subprocess execution is restricted or fails.

How does the server know which project to watch after indexing?

The register_watcher_if_enabled() function passes the project identifier generated by cbm_project_name_from_path to cbm_watcher_watch. This registration associates the specific project path with the global watcher instance, ensuring subsequent filesystem events in that directory trigger the correct re-index callback.

Can I disable automatic re-indexing while keeping the initial auto-index feature?

Yes. Set the auto_watch configuration flag to false using codebase-memory-mcp config set auto_watch false. This prevents register_watcher_if_enabled() from calling cbm_watcher_watch, disabling the filesystem polling while still allowing the initial background index to run when auto_index is true.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}