# How Dump Verification Ensures Data Integrity After Indexing in codebase-memory-mcp

> Learn how dump verification in codebase-memory-mcp safeguards data integrity by comparing in-memory node counts with persisted SQLite rows, detecting data loss.

- Repository: [Martin Vogel/codebase-memory-mcp](https://github.com/DeusData/codebase-memory-mcp)
- Tags: how-to-guide
- Published: 2026-07-11

---

**The codebase-memory-mcp repository implements a post-dump plausibility gate that compares committed in-memory node counts against actual persisted SQLite rows, flagging data loss when the ratio falls below a configurable threshold.**

After `index_repository` completes, the Memory-Code-Profiler (MCP) must confirm that the SQLite database accurately reflects the indexed data. According to the DeusData/codebase-memory-mcp source code, the system performs a lightweight sanity-check via [`src/foundation/dump_verify.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/foundation/dump_verify.c) that acts as a final gate before returning success to the client.

## The Post-Dump Plausibility Gate

The verification process follows a pipeline to detect substantial data loss before the JSON response is returned.

### Gathering Expected Counts

When indexing finishes, the pipeline reports the number of nodes and edges it committed in memory via `cbm_pipeline_get_committed_counts`. This establishes the baseline expectation for what should exist in the database.

### Reading the Persisted Store

The MCP opens the per-project SQLite store and counts actual rows using `cbm_store_count_nodes` and `cbm_store_count_edges`. These functions query the persisted state in [`src/mcp/mcp.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/mcp/mcp.c) to determine what actually survived the write operation.

### Applying the Ratio Gate

The core verification logic resides in `cbm_dump_verify_is_degraded` in [`src/foundation/dump_verify.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/foundation/dump_verify.c). This function compares persisted counts against committed counts using a configurable ratio threshold. The default minimum ratio is **0.5** (50%), read from the environment variable `CBM_DUMP_VERIFY_MIN_RATIO`. The gate automatically disables itself for very small repositories (≤ 50 nodes) or when the ratio is set to ≤ 0.

### Recovery and Re-verification

If the initial check flags degradation, the system attempts self-healing. The store is checkpointed via `cbm_store_checkpoint`, counts are re-read, and the gate is re-evaluated. This handles transient corruption or stale reads that might occur during the initial verification.

### Reporting the Outcome

When degradation is confirmed, the JSON response includes a `hint` field explaining the likely cause (such as a hard-killed sibling process), and a warning is logged via `cbm_log_warn`. If the store is missing entirely, a different hint is provided instructing the caller to re-run `index_repository`.

## Key Implementation Details

The verification system spans multiple files with specific responsibilities.

### The Gate Implementation

In [`src/foundation/dump_verify.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/foundation/dump_verify.c), the `cbm_dump_verify_is_degraded` function implements the plausibility logic:

```c
bool cbm_dump_verify_is_degraded(int committed_nodes, int persisted_nodes,
                                 double ratio, int min_floor) {
    if (ratio <= 0.0)                return false;   // gate disabled
    if (committed_nodes < 0)        return false;   // no dump performed
    if (committed_nodes <= min_floor) return false; // tiny repo – skip
    if (persisted_nodes < 0)        return true;    // error reading count
    return (double)persisted_nodes < (double)committed_nodes * ratio;
}

```

### Configuring the Verification Threshold

The ratio threshold is configurable via environment variables, parsed in `cbm_dump_verify_min_ratio`:

```c
double cbm_dump_verify_min_ratio(void) {
    char buf[CBM_SZ_32];
    if (cbm_safe_getenv("CBM_DUMP_VERIFY_MIN_RATIO", buf, sizeof(buf), NULL) != NULL) {
        char *end = NULL;
        double r = strtod(buf, &end);
        if (end != buf && r >= 0.0 && r <= 1.0) return r;
        cbm_log_warn("dump_verify.env.invalid", "value", buf, "fallback", "0.5");
    }
    return CBM_DUMP_VERIFY_DEFAULT_RATIO; // 0.5
}

```

### Integration in the Indexing Pipeline

The verification gate is invoked in [`src/mcp/mcp.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/mcp/mcp.c) after the indexing pipeline completes:

```c
int exp_nodes = -1, exp_edges = -1;
cbm_pipeline_get_committed_counts(p, &exp_nodes, &exp_edges);

const double ratio = cbm_dump_verify_min_ratio();
const int min_floor = CBM_DUMP_VERIFY_MIN_FLOOR;

cbm_store_t *store = resolve_store(srv, project_name);
int nodes = 0, edges = 0;
bool degraded = false;

if (store) {
    nodes = cbm_store_count_nodes(store, project_name);
    edges = cbm_store_count_edges(store, project_name);
    if (cbm_dump_verify_is_degraded(exp_nodes, nodes, ratio, min_floor)) {
        (void)cbm_store_checkpoint(store);          // flush & reopen
        nodes = cbm_store_count_nodes(store, project_name);
        edges = cbm_store_count_edges(store, project_name);
        degraded = cbm_dump_verify_is_degraded(exp_nodes, nodes, ratio, min_floor);
    }
}

```

### User-Facing Error Reporting

When verification fails, the system adds actionable hints to the JSON response:

```c
if (degraded) {
    yyjson_mut_obj_add_str(doc, root, "hint",
        "Persisted far fewer nodes than indexed — likely durability loss from a "
        "hard‑killed sibling process. Re‑run index_repository(...) to rebuild.");
    cbm_log_warn("dump.verify", "expected_nodes", exp_buf,
                 "persisted_nodes", got_buf);
}

```

## Summary

- **Ratio-based checking**: The system compares committed node counts against persisted rows using a configurable threshold (default 0.5) to detect substantial data loss.
- **Automatic bypass for small repos**: Repositories with ≤ 50 nodes skip verification to prevent false positives on tiny codebases.
- **Self-healing attempt**: If initial verification fails, the system checkpoints the store and re-verifies before marking the dump as degraded.
- **Clear failure signaling**: Degraded states trigger JSON hints and log warnings, instructing users to re-run `index_repository` when data integrity is compromised.

## Frequently Asked Questions

### What is the default minimum ratio for dump verification?

The default minimum ratio is **0.5** (50%), meaning the system tolerates up to 50% data loss before flagging degradation. This value is defined as `CBM_DUMP_VERIFY_DEFAULT_RATIO` in [`src/foundation/dump_verify.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/foundation/dump_verify.c) and can be overridden via the `CBM_DUMP_VERIFY_MIN_RATIO` environment variable.

### Why does verification skip small repositories?

Repositories with **50 or fewer nodes** bypass the ratio gate to prevent false positives. Tiny codebases can exhibit statistical anomalies during persistence that don't indicate actual corruption, so the `min_floor` parameter (set to 50) disables the check for these cases.

### How does the system handle transient corruption during verification?

If the initial check detects degradation, the MCP performs a **checkpoint and re-verification cycle**. It calls `cbm_store_checkpoint` to flush and reopen the database, then re-counts the nodes and edges. This handles transient states or stale reads that might occur immediately after the initial write.

### What happens when dump verification fails?

When verification fails, the JSON response includes a `hint` field explaining that "persisted far fewer nodes than indexed" were stored, typically indicating a hard-killed sibling process. A warning is logged via `cbm_log_warn`, and the caller is instructed to re-run `index_repository` to rebuild the database with correct data.