# How codebase-memory-mcp Builds a Knowledge Graph from Source Code: A Deep Dive into the Three-Stage Pipeline

> Learn how codebase-memory-mcp builds a knowledge graph from source code using a three-stage pipeline. Parse code with Tree-sitter, buffer symbols, and store in SQLite.

- Repository: [Martin Vogel/codebase-memory-mcp](https://github.com/DeusData/codebase-memory-mcp)
- Tags: deep-dive
- Published: 2026-07-15

---

**The `codebase-memory-mcp` tool constructs a persistent knowledge graph by parsing source code with Tree-sitter, creating an in-memory graph buffer of symbols and relationships, and flushing the results to a SQLite database for fast structured queries.**

The `codebase-memory-mcp` project transforms raw source trees into queryable graph databases that power AI-assisted code understanding. By combining Tree-sitter parsing with a custom two-phase extraction pipeline, this Model Context Protocol (MCP) server enables developers to build a knowledge graph from source code that persists in SQLite and supports complex structural queries without re-parsing files.

## The Three-Stage Knowledge Graph Construction Pipeline

The architecture follows a strict three-stage process: extraction, in-memory graph construction, and SQLite persistence. Each stage is decoupled yet tightly integrated to handle repositories containing millions of lines of code.

### Stage 1: Source Code Extraction with Tree-sitter

The pipeline begins in [`src/discover/discover.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/discover/discover.c), which walks the repository tree and filters files by supported languages (158 bundled Tree-sitter grammars). For each eligible file, the system calls `cbm_extract_file` to parse the source into a `CBMFileResult` structure.

This result container captures:
- **Definitions**: functions, classes, methods, variables
- **Call sites**: function invocations, HTTP routes, async boundaries
- **Imports**: module dependencies and package references
- **Channels**: event emissions and listeners
- **Environment accesses**: configuration reads

The extraction logic spans [`src/pipeline/pass_definitions.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/pipeline/pass_definitions.c) (for symbol definitions) and [`src/pipeline/pass_calls.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/pipeline/pass_calls.c) (for call relationships), utilizing Tree-sitter ASTs enhanced by optional LSP semantic data from [`src/semantic/semantic.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/semantic/semantic.c).

### Stage 2: Node and Edge Creation

Once extracted, the system builds an in-memory representation using the **graph buffer** implemented in [`src/graph_buffer/graph_buffer.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/graph_buffer/graph_buffer.c). This stage operates in two distinct phases orchestrated by [`src/pipeline/pipeline.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/pipeline/pipeline.c):

**Phase 1: Definitions Pass**
- `cbm_pipeline_pass_definitions` processes each `CBMFileResult`
- `process_def` constructs JSON property blobs via `build_def_props`
- `cbm_gbuf_upsert_node` creates nodes (`cbm_node_t`) for every symbol with qualified names
- `cbm_registry_add` registers each symbol in a central registry for later resolution
- `"DEFINES"` edges connect file nodes (`__file__`) to their contained symbols

**Phase 2: Resolution Pass**
- After caching results (`result_cache`), the pipeline builds a namespace map via `cbm_pipeline_namespace_map_build`
- `create_import_edges_for_file` resolves imports using `cbm_pipeline_resolve_import_node` and creates `"IMPORTS"` edges
- [`cbm_pipeline_pass_calls.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/cbm_pipeline_pass_calls.c) creates `"CALLS"`, `"HTTP_CALLS"`, and `"ASYNC_CALLS"` edges linking call sites to definitions
- Channel edges (`"EMITS"`, `"LISTENS_ON"`) and configuration edges (`"CONFIGURES"`) bind runtime behavior to static symbols

### Stage 3: Persistence to SQLite

The final stage converts the mutable graph buffer into a durable SQLite database stored in `~/.cache/codebase-memory-mcp`. The storage layer in [`src/store/store.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/store/store.c) implements bulk write operations optimized for large graphs:

- `cbm_store_begin_bulk` initiates a transaction
- `cbm_store_upsert_node_batch` persists nodes (`cbm_node_t`)
- `cbm_store_insert_edge_batch` persists relationships (`cbm_edge_t`)
- `cbm_store_check_integrity` validates the database; corrupted files are renamed to `*.corrupt`

The schema defined in [`src/store/store.h`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/store/store.h) supports compressed artifacts (`.zst`) for fast team bootstrapping, allowing new team members to download pre-indexed graphs rather than re-parsing entire repositories.

## From MCP Request to Structured Graph

When an AI agent invokes the `index_repository` tool, the request flows through [`src/mcp/mcp.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/mcp/mcp.c) where `cbm_jsonrpc_parse` handles JSON-RPC dispatch. The server validates arguments via `cbm_mcp_get_string_arg` and initializes a `cbm_mcp_server_t` instance holding the project store.

The indexing workflow proceeds as follows:

1. **Discovery**: [`discover.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/discover.c) enumerates source files and builds a `cbm_file_info_t` array
2. **Extraction**: `cbm_extract_file` generates `CBMFileResult` structures containing AST-derived metadata
3. **Node Creation**: The definitions pass upserts nodes into the graph buffer and populates the symbol registry
4. **Edge Resolution**: Import and call edges are resolved against the fully populated registry
5. **Persistence**: The graph buffer flushes to SQLite via batched upsert operations

## Querying the Knowledge Graph

Once persisted, the graph supports high-performance queries without re-parsing source files. MCP tools exposed in [`src/mcp/mcp.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/mcp/mcp.c) include:

- **`search_graph`**: Executes BM25 full-text search via `cbm_store_search` with optional vector similarity (`cbm_store_vector_search`)
- **`trace_path`**: Performs breadth-first traversal (`cbm_store_bfs`) over `"CALLS"` edges to map dependency chains
- **`query_graph`**: Supports Cypher graph queries (`cbm_cypher_execute`) against the SQLite-backed schema

The result is a **single-file persistent graph** that enables AI agents to answer structural questions with approximately 120× token reduction compared to naive text searching.

## Code Examples

### Index a Repository via CLI

```bash

# Index the current repo and write a compressed artifact for teammates

codebase-memory-mcp index_repository \
    --repo_path . \
    --persistence true \
    --name my-project

```

This command routes through [`src/cli/cli.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/cli/cli.c) to issue an MCP `index_repository` request handled in [`src/mcp/mcp.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/mcp/mcp.c).

### Search the Graph via JSON-RPC

```json
{
  "jsonrpc": "2.0",
  "method": "search_graph",
  "id": 1,
  "params": {
    "project": "my-project",
    "query": "publish event",
    "limit": 5
  }
}

```

The server executes `cbm_store_search` with full-text ranking and returns relevance-scored nodes.

### Trace a Call Chain

```json
{
  "jsonrpc": "2.0",
  "method": "trace_path",
  "id": 2,
  "params": {
    "function_name": "UserService.sendMessage",
    "project": "my-project",
    "direction": "outbound",
    "depth": 3,
    "mode": "calls"
  }
}

```

The handler invokes `cbm_store_bfs` to traverse `"CALLS"` edges and returns `cbm_node_hop_t` structures showing the execution path.

## Summary

- **Tree-sitter parsing** extracts definitions, calls, and dependencies from source files into `CBMFileResult` structures
- **Two-phase pipeline** first creates symbol nodes and registers them, then resolves import and call edges using the registry
- **Graph buffer** ([`src/graph_buffer/graph_buffer.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/graph_buffer/graph_buffer.c)) provides fast mutable storage before persistence
- **SQLite backend** ([`src/store/store.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/store/store.c)) enables compressed, portable graph files with full-text and vector search capabilities
- **MCP integration** exposes graph construction and querying via standardized JSON-RPC tools for AI agent integration

## Frequently Asked Questions

### What database does codebase-memory-mcp use for the knowledge graph?

The system uses **SQLite** as its persistence layer, storing nodes (`cbm_node_t`) and edges (`cbm_edge_t`) in a relational schema defined in [`src/store/store.h`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/store/store.h). The database supports optional Zstandard compression (`.zst`) and is typically cached under `~/.cache/codebase-memory-mcp`.

### How does the tool handle multiple programming languages?

The extraction layer utilizes **Tree-sitter** with 158 bundled grammars to parse diverse languages. File discovery in [`src/discover/discover.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/discover/discover.c) filters repositories by language support, routing each file to the appropriate parser while normalizing outputs into the common `CBMFileResult` format.

### What types of relationships does the graph capture?

Beyond standard `"DEFINES"` (file-to-symbol) and `"CALLS"` edges, the graph captures `"IMPORTS"` for module dependencies, `"HTTP_CALLS"` for API endpoints, `"ASYNC_CALLS"` for asynchronous boundaries, `"EMITS"`/`"LISTENS_ON"` for event-driven channels, and `"CONFIGURES"` for environment variable access.

### Can the knowledge graph be shared among team members?

Yes. The SQLite database can be compressed using Zstandard (`.zst`) to create portable artifacts. Teams can distribute these compressed graph files rather than requiring each developer to re-parse large repositories, significantly reducing bootstrapping time for new team members.