Performance Characteristics of Codebase Memory MCP: Benchmarks, Architecture, and Optimization

Codebase Memory MCP achieves sub-millisecond query response times and indexes the entire Linux kernel in approximately three minutes by combining a RAM-first pipeline, in-memory SQLite, LZ4 compression, and a highly optimized Aho-Corasick matcher.

The DeusData/codebase-memory-mcp repository implements a lightweight, single-binary engine designed for massive-scale code analysis. Understanding the performance characteristics of codebase memory MCP reveals how it maintains extreme indexing speeds while keeping memory usage modest, making it suitable for everything from small repositories to the Linux kernel.

Extreme Indexing Throughput

Codebase Memory MCP is engineered for extremely fast indexing across massive codebases. The system indexes the Linux kernel—approximately 28 million lines of code across 75,000 files—in roughly three minutes, as documented in the repository README.

For typical repositories, the process completes in milliseconds, enabling near-instantaneous startup for most development workflows. This performance is achieved through a combination of vendored Tree-Sitter grammars for direct AST parsing and LZ4 compression to minimize RAM consumption during ingestion.

Sub-Millisecond Query Latency

Query performance represents a core optimization target for the engine. Structural (Cypher-like) queries complete in less than 1 millisecond, providing real-time navigation of code relationships.

This latency bound applies consistently across the supported query types, including complex graph traversals that map function calls, imports, and dependencies. The performance remains stable regardless of repository size, as the underlying in-memory SQLite database avoids disk I/O bottlenecks.

Memory Footprint and Efficiency

Despite handling massive codebases, the binary maintains a modest memory profile. During a soak test on macOS processing 256 continuous queries, the RSS remained around 18 MiB with a fixed heap of 15 MiB, as recorded in soak-ql-mac-fixed/metrics.csv.

This efficiency stems from the RAM-first pipeline architecture, which uses LZ4 compression for raw source storage and in-memory SQLite virtual tables rather than disk-backed databases. The result is a lightweight process suitable for long-running development environments and resource-constrained systems.

Cross-Language Benchmark Coverage

The project ships a comprehensive benchmark suite measuring indexing quality across 158 programming languages. Performance varies by language complexity and Tree-Sitter grammar maturity:

Tier	Languages	Accuracy	Scale
Tier 1 (≥90%)	Lua, Kotlin, C++, Perl, Objective-C, Groovy, C, Bash, Zig, Swift, CSS, YAML, HTML	100%	23k–49k nodes
Tier 2 (75–89%)	Python, TypeScript, Go, Rust, Java, R, Dart, JavaScript, Erlang, Elixir, Scala, Ruby, PHP, C#, SQL	87%–95%	2k–49k nodes
Tier 3 (<75%)	OCaml (72%), Haskell (62%)	Variable	Variable

Full benchmark methodologies and detailed results are available in docs/BENCHMARK.md.

Architectural Optimizations

The performance characteristics of codebase memory MCP rely on several tightly integrated components:

Tree-Sitter Grammars

Vendored Tree-Sitter grammars parse source code into ASTs for every supported language. This eliminates external parser dependencies and allows the binary to directly walk syntax trees without intermediate representations.

In-Memory SQLite with FTS5

The engine uses in-memory SQLite configured with FTS5 for full-text search. By leveraging SQLite's virtual table interface as a memory-resident store, the system avoids disk I/O entirely during queries.

LZ4 Compression

Raw source files undergo LZ4 compression before indexing. This fast, block-level compression reduces the volume of data kept in RAM while maintaining CPU efficiency during decompression.

Fused Aho-Corasick Matcher

A highly optimized Aho-Corasick matcher performs linear-time pattern matching across the entire codebase. This algorithm enables rapid token-level indexing and query resolution with guaranteed O(n) performance characteristics.

Zstandard Persistence

The internal/cbm/zstd_store.c implementation provides Zstandard ("zstd") compression for persistent graph snapshots. This allows occasional dumps of the in-memory graph with high compression ratios and low CPU overhead.

Coordination Daemon (CBM)

The CBM (Codebase Memory) daemon manages shared state, background file watchers, and UI processes. By guaranteeing a single running binary version, it prevents redundant indexing work and coordinates incremental updates via src/watcher/watcher.c.

Practical CLI Usage

All operations run locally without external API keys or network calls:

# Index the current repository (full-binary, no UI)

codebase-memory-mcp index .

# Retrieve a high-level architecture summary

codebase-memory-mcp get_architecture

# Run a Cypher-style query in <1 ms

codebase-memory-mcp query "MATCH (f:Function)-[:CALLS]->(g) WHERE f.name='main' RETURN g.name"

# Perform semantic vector search using bundled Nomic embeddings

codebase-memory-mcp semantic_query "authentication middleware"

Key Source Files

Path	Description
internal/cbm/zstd_store.c	Zstandard-based persistent store for graph snapshots
src/watcher/watcher.c	File-system watcher for incremental indexing
scripts/memlab.sh	Benchmark suite runner and metrics collector
graph-ui/vite.config.ts	Configuration for the optional 3-D UI variant
docs/BENCHMARK.md	Comprehensive language benchmark results
soak-ql-mac-fixed/metrics.csv	Long-running memory usage validation data

Summary

• Indexing Speed: The Linux kernel (~28M LOC) indexes in ~3 minutes; average repositories take milliseconds.
• Query Latency: Structural Cypher-like queries complete in sub-millisecond timeframes (<1 ms).
• Memory Usage: Stable ~18 MiB RSS during heavy query loads with 15 MiB fixed heap allocation.
• Language Support: Optimized parsing for 158 languages using vendored Tree-Sitter grammars.
• Architecture: RAM-first pipeline combining in-memory SQLite, LZ4 compression, and Aho-Corasick matching.
• Deployment: Single binary with no external dependencies or network requirements.

Frequently Asked Questions

What is the indexing speed for large repositories?

Codebase Memory MCP indexes the Linux kernel—approximately 28 million lines of code across 75,000 files—in roughly three minutes. Typical repositories complete indexing in milliseconds, enabling real-time development workflows without startup delays.

How much RAM does codebase memory MCP require?

During soak testing with 256 continuous queries, the binary maintained approximately 18 MiB RSS with a 15 MiB fixed heap. This modest footprint results from LZ4 compression of source files and the use of in-memory SQLite rather than disk-backed storage.

Which programming languages have the highest parsing accuracy?

Tier 1 languages—including Lua, Kotlin, C++, Perl, Objective-C, C, Bash, Zig, Swift, and markup languages like CSS, YAML, and HTML—achieve 100% accuracy scores. Python, TypeScript, Go, Rust, and Java typically score between 87% and 95%, while functional languages like OCaml and Haskell currently fall below 75%.

How does the engine achieve sub-millisecond query times?

The combination of an in-memory SQLite database with FTS5, a fused Aho-Corasick matcher providing linear-time pattern matching, and aggressive LZ4 compression eliminates disk I/O and minimizes CPU overhead. According to the source implementation, structural queries traverse pre-computed graph edges stored in RAM, avoiding the latency penalties associated with traditional disk-based code search tools.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}