Codebase Memory MCP Modules: Complete Architecture and Source Guide

TLDR: The codebase-memory-mcp repository organizes its functionality into 12 distinct modules, including a high-performance C Core Engine (CBM), Tree-sitter language grammars, symbol extraction modules, LSP back-ends, Python wrappers, and a web-based Graph UI, each serving specialized roles in the indexing pipeline.

The codebase-memory-mcp project, hosted at DeusData/codebase-memory-mcp, is a language-agnostic code intelligence engine designed for AI-assisted coding agents. Its architecture spans from low-level C libraries that parse source files to high-level Python interfaces and interactive web visualizations. Understanding the different codebase memory MCP modules reveals how the system extracts symbols, exposes data to editors, and provides command-line tooling.

Core Engine (CBM) and Language Grammars

At the heart of the system lies the Core Engine (CBM), a high-performance C library that drives the indexing pipeline. According to the DeusData/codebase-memory-mcp source code, the central implementation resides in [internal/cbm/cbm.c](https://github.com/DeusData/codebase-memory-mcp/blob/main/internal/cbm/cbm.c), with public API definitions in [internal/cbm/cbm.h](https://github.com/DeusData/codebase-memory-mcp/blob/main/internal/cbm/cbm.h). This engine parses source files and manages the overall extraction workflow.

The engine loads Language Grammars implemented as Tree-sitter wrappers. Each supported language lives in a separate C file within the same directory, such as [internal/cbm/grammar_python.c](https://github.com/DeusData/codebase-memory-mcp/blob/main/internal/cbm/grammar_python.c) for Python parsing. The repository contains approximately 80 grammar files covering languages including Java (grammar_java.c), Go (grammar_go.c), and YAML (grammar_yaml.c), enabling the engine to produce abstract syntax trees for diverse codebases.

Extraction Modules and LSP Back-ends

Once the Core Engine generates an AST, Extraction Modules traverse the tree to emit structured entities. The [internal/cbm/extract_defs.c](https://github.com/DeusData/codebase-memory-mcp/blob/main/internal/cbm/extract_defs.c) module handles definition extraction, while [internal/cbm/extract_calls.c](https://github.com/DeusData/codebase-memory-mcp/blob/main/internal/cbm/extract_calls.c) captures function calls, imports, and usages. These modules convert raw parse trees into typed nodes that the system can index and query.

LSP Back-ends expose this extracted data to editors and IDEs through Language Server Protocol adapters. Language-specific implementations include [internal/cbm/lsp/py_lsp.c](https://github.com/DeusData/codebase-memory-mcp/blob/main/internal/cbm/lsp/py_lsp.c) for Python support and [internal/cbm/lsp/java_lsp.c](https://github.com/DeusData/codebase-memory-mcp/blob/main/internal/cbm/lsp/java_lsp.c) for Java integration. These adapters enable IDE features like "go-to-definition" and "find-references" by translating the engine's internal symbol table into LSP-compliant messages.

Python Package and CLI Interface

The Python Package (codebase_memory_mcp) provides a thin wrapper around the compiled C binary. Located in pkg/pypi/src/codebase_memory_mcp/, this module ships the core engine as a binary artifact and exposes it through Python APIs. The [__init__.py](https://github.com/DeusData/codebase-memory-mcp/blob/main/pkg/pypi/src/codebase_memory_mcp/__init__.py) initializes the package, while [_cli.py](https://github.com/DeusData/codebase-memory-mcp/blob/main/pkg/pypi/src/codebase_memory_mcp/_cli.py) implements the command-line interface.

The Command-Line Interface module handles argument parsing, sub-commands, and output formatting. Users invoke the cbm tool through the Python package entry point defined in __main__.py, which delegates to the underlying C engine and formats results for terminal display or JSON consumption.

Preprocessor and Foundation Utilities

Supporting the Core Engine are Preprocessor & Foundations modules that handle low-level concerns. The [internal/cbm/preprocessor.c](https://github.com/DeusData/codebase-memory-mcp/blob/main/internal/cbm/preprocessor.c) file manages source preprocessing, while utilities in [internal/cbm/foundation/hash_table.h](https://github.com/DeusData/codebase-memory-mcp/blob/main/internal/cbm/foundation/hash_table.h) provide memory arenas and hash table implementations. These foundation modules ensure consistent memory management and cross-platform compatibility across the C codebase.

Graph UI and Visualization

The Graph UI module offers a web-based interface for visualizing indexed codebases. Built with Vite and TypeScript, the source resides in the graph-ui/ directory. Configuration files like [vite.config.ts](https://github.com/DeusData/codebase-memory-mcp/blob/main/graph-ui/vite.config.ts) and [tsconfig.json](https://github.com/DeusData/codebase-memory-mcp/blob/main/graph-ui/tsconfig.json) define the build environment. This module consumes JSON outputs from the Core Engine to render interactive dependency graphs and enable exploratory queries through a browser interface.

Testing, Scripts, and Build Infrastructure

Scripts & Utilities provide development automation and quality assurance. The [scripts/memlab-report.py](https://github.com/DeusData/codebase-memory-mcp/blob/main/scripts/memlab-report.py) generates memory profiling reports, while [scripts/license-gate-check.py](https://github.com/DeusData/codebase-memory-mcp/blob/main/scripts/license-gate-check.py) validates licensing compliance across dependencies.

The Test Suite ensures correctness across platforms with comprehensive automated coverage. Key tests include [tests/test_daemon_smoke.py](https://github.com/DeusData/codebase-memory-mcp/blob/main/tests/test_daemon_smoke.py) for daemon lifecycle validation and [tests/windows/test_non_ascii_path.py](https://github.com/DeusData/codebase-memory-mcp/blob/main/tests/windows/test_non_ascii_path.py) for Windows-specific path handling.

Build & Packaging modules handle compilation and distribution. The Makefile.cbm orchestrates C engine compilation, while [install.sh](https://github.com/DeusData/codebase-memory-mcp/blob/main/install.sh) provides repository setup automation. Documentation modules include configuration guides in [docs/CONFIGURATION.md](https://github.com/DeusData/codebase-memory-mcp/blob/main/docs/CONFIGURATION.md) and performance benchmarks in [docs/BENCHMARK.md](https://github.com/DeusData/codebase-memory-mcp/blob/main/docs/BENCHMARK.md).

Summary

• Core Engine (CBM): High-performance C library in internal/cbm/cbm.c that parses files and drives indexing.
• Language Grammars: Approximately 80 Tree-sitter wrappers (e.g., grammar_python.c) enabling multi-language AST generation.
• Extraction Modules: AST traversal code in extract_defs.c and extract_calls.c that emits structured symbols.
• LSP Back-ends: Editor adapters in lsp/py_lsp.c and java_lsp.c exposing symbols via Language Server Protocol.
• Python Package: Wrapper module shipping the compiled binary with CLI entry points in _cli.py.
• Preprocessor & Foundations: Memory arenas and hash tables in foundation/hash_table.h supporting the C engine's infrastructure.
• Graph UI: Vite-based web interface in graph-ui/ for interactive visualization of indexed codebases.
• Testing & Scripts: Cross-platform test suites in tests/ and utility scripts in scripts/ for benchmarking and license validation.
• Build System: Makefile-based compilation via Makefile.cbm and packaging scripts for distribution.

Frequently Asked Questions

What is the Core Engine (CBM) module in codebase-memory-mcp?

The Core Engine (CBM) is the central C library implemented in [internal/cbm/cbm.c](https://github.com/DeusData/codebase-memory-mcp/blob/main/internal/cbm/cbm.c) that parses source files using Tree-sitter grammars and drives the symbol indexing pipeline. It serves as the performance-critical foundation that extracts definitions, calls, and imports from codebases before exposing them to higher-level modules.

How does the Python package interact with the Core Engine?

The Python Package (codebase_memory_mcp) acts as a thin wrapper located in pkg/pypi/src/codebase_memory_mcp/. It ships the precompiled C binary and invokes it through the implementation in [_cli.py](https://github.com/DeusData/codebase-memory-mcp/blob/main/pkg/pypi/src/codebase_memory_mcp/_cli.py), then formats the engine's output for Python consumers. The entry point defined in __main__.py handles the cbm command-line invocation.

Which programming languages are supported by the grammar modules?

The Language Grammars module supports approximately 80 programming languages through individual Tree-sitter grammar files in internal/cbm/. Notable examples include [grammar_python.c](https://github.com/DeusData/codebase-memory-mcp/blob/main/internal/cbm/grammar_python.c) for Python, grammar_java.c for Java, grammar_go.c for Go, and [grammar_yaml.c](https://github.com/DeusData/codebase-memory-mcp/blob/main/internal/cbm/grammar_yaml.c) for YAML configuration files.

How is the Graph UI module used in the codebase-memory-mcp ecosystem?

The Graph UI module provides a web-based visualization layer built with Vite and TypeScript in the graph-ui/ directory. It consumes JSON outputs produced by the Core Engine to render interactive dependency graphs and symbol relationships, allowing developers to explore indexed codebases through a browser interface rather than command-line queries.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}