# What Is the Index Supervisor in Codebase-Memory-MCP and When Does It Spawn Child Processes?

> Discover the index supervisor in Codebase-Memory-MCP, a crash-isolation layer that spawns worker subprocesses for repository indexing only in the host MCP process, preventing main server crashes.

- Repository: [Martin Vogel/codebase-memory-mcp](https://github.com/DeusData/codebase-memory-mcp)
- Tags: internals
- Published: 2026-07-13

---

**The index supervisor is a crash-isolation layer that spawns supervised worker subprocesses to handle repository indexing only when running in the host MCP process, preventing individual file failures from crashing the main server.**

The Codebase-Memory-MCP server from DeusData uses the *index supervisor* to protect the long-running MCP process from crashes or hangs caused by problematic files during indexing. This lightweight orchestration layer ensures that the actual `index_repository` operation runs inside a separate binary process, isolating catastrophic failures from the main application.

## Architecture of the Index Supervisor

### Purpose and Design Goals

The index supervisor serves as a defensive barrier between the MCP server and the native indexing engine. Its primary responsibility is to execute the heavy indexing work inside a **supervised worker subprocess**—the same binary re-invoked with `cli --index-worker …` arguments—so that segmentation faults or infinite loops in the indexer never terminate the parent process.

This design prioritizes **resilience over performance**: while spawning a subprocess adds overhead, it guarantees that a single malformed file cannot bring down the entire MCP service.

### Host Marking and Safety Guards

To prevent recursive spawning and ensure only the real MCP binary can delegate work, the supervisor implements a host-marking protocol. In [`src/main.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/main.c) at approximately line 182, the real MCP binary calls `cbm_index_supervisor_mark_host()` at the start of `main()`:

```c
/* Mark this process as the host - only hosts may spawn workers */
cbm_index_supervisor_mark_host();   // src/main.c: ≈182

```

Embedded callers or library consumers never invoke this function, which prevents them from accidentally spawning workers. The supervisor tracks this state through internal globals (`g_host_marked` and `g_worker_active`) that distinguish between the host process and worker children.

## When Does the Index Supervisor Spawn Child Processes?

### Decision Logic in cbm_index_supervisor_should_wrap

The function `cbm_index_supervisor_should_wrap()` in [`src/mcp/index_supervisor.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/mcp/index_supervisor.c) (lines 73-86) determines whether to wrap the indexing call in a subprocess. It returns **true** only when all three conditions are met:

1. The process is the marked host (`g_host_marked` is set)
2. The process is **not** already an active worker (`g_worker_active` is false)
3. The kill-switch environment variable `CBM_INDEX_SUPERVISOR` is not set to `0`

This triple-check ensures that workers cannot spawn additional workers, and users can disable supervision entirely by setting `CBM_INDEX_SUPERVISOR=0`.

### Trigger Points in the Codebase

The supervisor is consulted at two critical points in the indexing path:

- **File system watcher**: In [`src/watcher/watcher.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/watcher/watcher.c), the `watcher_index_fn` callback triggers supervised indexing when repository changes are detected
- **MCP request handler**: In [`src/mcp/mcp.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/mcp/mcp.c) at approximately line 4855, the function `cbm_mcp_index_run_supervised_path` checks the supervisor before executing index commands

When `cbm_index_supervisor_should_wrap()` returns true, these paths invoke `cbm_index_spawn_worker()` to create the isolated process.

## How the Spawn Mechanics Work

### Building the Worker Arguments

The `cbm_index_spawn_worker()` function in [`src/mcp/index_supervisor.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/mcp/index_supervisor.c) (lines 58-88 and 147-188) constructs a specific argument array to re-execute the current binary as a worker:

```c
int cbm_index_spawn_worker(const char *args_json, bool single_thread,
                           const char *marker_file, const char *quarantine_file,
                           cbm_index_worker_result_t *result) {
    /* Build argv for the child process */
    const char *argv[8];
    int n = 0;
    argv[n++] = self;               // resolved binary path
    argv[n++] = "cli";
    argv[n++] = "--index-worker";
    argv[n++] = "index_repository";
    argv[n++] = args_json;
    argv[n++] = "--response-out";
    argv[n++] = resp_path;
    argv[n]   = NULL;
    
    /* fork + exec performed by cbm_proc_spawn ... */
}

```

The function resolves the binary path, creates temporary files for the worker's response and logs, sets optional probe environment variables, and then calls `fork` followed by `exec`.

### Process Creation and Fallback Behavior

If the spawn succeeds, the parent process waits to reap the child, reads the response from the temporary file if the exit was clean, and continues operation. However, if spawning fails—for example, if the binary path cannot be resolved—the supervisor **degrades gracefully** to an in-process indexing run (see [`src/main.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/main.c) lines 92-94). This fallback ensures that indexing remains functional even when process supervision is unavailable due to system constraints.

## Summary

- The **index supervisor** isolates indexing crashes by running `index_repository` in a separate subprocess
- It only activates in the **host MCP process** (marked via `cbm_index_supervisor_mark_host()`) and never inside existing workers
- Spawning occurs when repository changes are detected by the watcher or during MCP request handling, provided the `CBM_INDEX_SUPERVISOR` environment variable is not disabled
- The worker process receives arguments via `cli --index-worker index_repository` with JSON configuration and writes responses to temporary files
- Failed spawns automatically fall back to in-process execution to maintain availability

## Frequently Asked Questions

### What happens if the index supervisor cannot spawn a worker process?

If `cbm_index_spawn_worker()` fails to resolve the binary path or execute the fork operation, the supervisor degrades to in-process execution. As implemented in [`src/main.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/main.c) lines 92-94, the code detects the spawn failure and proceeds to run the indexing operation directly within the parent process, ensuring that repository indexing continues uninterrupted even when subprocess isolation is unavailable.

### How does the index supervisor prevent recursive spawning?

The supervisor uses a state flag `g_worker_active` that is set when the current process is operating as a worker. The function `cbm_index_supervisor_should_wrap()` explicitly checks this flag and returns false if `g_worker_active` is true, preventing workers from spawning additional workers. Additionally, only processes that have called `cbm_index_supervisor_mark_host()`—typically done only in the real MCP binary's `main()` function—are permitted to spawn, while embedded library callers never set this mark.

### Can I disable the index supervisor behavior?

Yes, you can disable the supervisor by setting the environment variable `CBM_INDEX_SUPERVISOR` to `0`. When this kill-switch is detected, `cbm_index_supervisor_should_wrap()` returns false regardless of other state conditions, forcing all indexing operations to run in-process without subprocess isolation.

### How can I monitor how many times the supervisor has spawned workers?

The supervisor exposes test hooks via `cbm_index_supervisor_spawn_count()` and `cbm_index_supervisor_spawn_st_count()` (defined in [`src/mcp/index_supervisor.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/mcp/index_supervisor.c) lines 50-64). These functions return the number of spawn attempts made, allowing the test suite to verify that embedding callers never trigger spawns and helping operators monitor indexing activity in production environments.