How the File Discovery Module Handles .cbmignore and .gitignore Precedence in Codebase Memory MCP

The file discovery module applies a strict five-layer cascade where repository .gitignore rules always take precedence over .cbmignore, while .cbmignore negations can only rescue files from Git global excludes.

The codebase-memory-mcp repository implements a deterministic ignore-layer system for its C-based file discovery engine. Understanding how .cbmignore and .gitignore precedence works is essential for configuring which files get indexed across your projects, as the order determines whether a negation pattern can actually include a previously excluded file.

The Five-Layer Precedence Hierarchy

According to the documentation in docs/cbmignore.md (lines 64‑83), the discovery engine evaluates ignore rules in the following order. The first layer that rejects a path wins immediately, stopping further evaluation.

1. Built-in skip list — Hard-coded directories like .git, node_modules, and dist that are never overridable.
2. Repository .gitignore — The root .gitignore file merged with Git’s info/exclude.
3. Nested .gitignore files — Additional .gitignore files discovered while walking subdirectories, matched relative to their own location.
4. .cbmignore — Project-specific ignore file where positive matches skip files, and negated matches (!) can rescue paths only from the next layer (Git global excludes).
5. Git global excludes — Patterns from core.excludesFile or XDG config directories.

This hierarchy ensures that local project configuration (layers 1‑3) remains authoritative over the custom .cbmignore layer, while .cbmignore serves as a mechanism to override user-level Git settings rather than repository-level rules.

Implementation in the Discovery Engine

Loading the Ignore Files

In src/discover/discover.c, the engine loads ignore files in strict sequence to prepare the cascade. The code first initializes the repository-level Git ignore, then the optional .cbmignore, and finally the global excludes:

/* src/discover/discover.c */

/* Load repository .gitignore (merged with info/exclude) */
cbm_gitignore_t *gitignore = cbm_gitignore_load(gi_path);  // line 741

/* Load optional .cbmignore supplied via CLI or at repo root */
cbm_gitignore_t *cbmignore = cbm_gitignore_load(opts->ignore_file);  // line 1040
if (!cbmignore) {
    cbmignore = cbm_gitignore_load(gi_path);  // line 1043
}

/* Git global excludes loaded separately (line 1018) */

The cbm_gitignore_t structures declared in src/discover/discover.h (lines 121‑150) store the compiled patterns for each layer, allowing the walker to query them in order.

Enforcing the Cascade

During the directory traversal, the function should_skip_directory() — called from walk_dir_process_entry at line 780 — implements the precedence logic. The walker checks each layer sequentially:

• Built-in skip list (hard-coded rejection).
• Repository .gitignore result.
• Nested .gitignore results discovered during the walk.
• .cbmignore result, where negative matches only reverse decisions from layer 5.
• Git global excludes (final layer).

Because the function returns immediately upon the first "skip" determination, the cascade achieves its first-match-wins semantics, ensuring that .gitignore (layer 2) blocks files before .cbmignore (layer 4) can attempt to rescue them.

Practical Precedence Examples

Rescuing Files from Global Excludes

When you need to track files that your global Git configuration normally ignores, .cbmignore can override layer 5:

# ~/.config/git/ignore (global exclude)

*.sql

# .cbmignore at repo root

!*.sql

Effect: The discovery module indexes *.sql files because the .cbmignore negation overrides the global exclude. This works because layer 4 is evaluated after layer 5 in the rescue context.

When .gitignore Overrides .cbmignore

Attempting to rescue a file blocked by the repository .gitignore fails because layer 2 precedes layer 4:

# repo/.gitignore

secret.conf

# repo/.cbmignore

!secret.conf

Effect: secret.conf remains excluded. The repository .gitignore is evaluated before .cbmignore, so the negation cannot override it.

Built-in Skips Are Absolute

The built-in skip list (layer 1) cannot be negated by any configuration:

# repo/.cbmignore

!node_modules/

Effect: node_modules remains skipped despite the negation pattern. Built-in entries are non-negatable safeguards that ignore all rescue attempts from .cbmignore.

Summary

• The five-layer cascade processes ignores in the order: built-in skips → repository .gitignore → nested .gitignore → .cbmignore → global Git excludes.
• .cbmignore negations (!) only function to rescue files from Git global excludes (layer 5), never from repository .gitignore or built-in skips.
• Built-in skips for directories like .git and node_modules are hard-coded in layer 1 and are absolute.
• The implementation in src/discover/discover.c enforces this order through sequential checks in should_skip_directory(), with the cbm_gitignore_t structures maintaining state for each precedence level.

Frequently Asked Questions

Can .cbmignore override a rule in my repository's .gitignore?

No. The repository .gitignore occupies layer 2, while .cbmignore occupies layer 4 in the precedence stack. Because the discovery engine stops at the first matching exclusion, a file rejected by .gitignore never reaches the .cbmignore evaluation stage. Negation patterns in .cbmignore can only reverse exclusions originating from Git global excludes at layer 5.

What happens if I try to negate a built-in skip like node_modules?

The built-in skip list (layer 1) is non-negatable. Even if you add !node_modules/ to your .cbmignore, the directory remains excluded because the cascade terminates at the hard-coded skip before any ignore-file logic executes. These built-in patterns protect critical directories from accidental indexing.

How does the discovery module handle nested .gitignore files?

Nested .gitignore files are evaluated as layer 3, after the repository root .gitignore (layer 2) but before .cbmignore (layer 4). Patterns in these files are matched relative to their containing directory, following standard Git semantics. This allows subdirectory-specific rules to take precedence over root-level patterns but still yields to .cbmignore and global excludes.

Where is the precedence order documented in the source code?

The precedence rules are documented in lines 64‑83 of docs/cbmignore.md, which describes the five-layer hierarchy and rescue semantics. The implementation resides in src/discover/discover.c, specifically within the should_skip_directory() function and the ignore-loading logic around lines 741‑1043 that initializes the cbm_gitignore_t structures declared in src/discover/discover.h.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}