# How Code-Graph-RAG's Tree-sitter Parser Extracts Code Structure into Memgraph

> Discover how Code-Graph-RAG's Tree-sitter parser extracts code structure into Memgraph. Learn about its three-pass pipeline for parsing, definition extraction, and streaming data.

- Repository: [Vitali Avagyan/code-graph-rag](https://github.com/vitali87/code-graph-rag)
- Tags: how-to-guide
- Published: 2026-08-13

---

**Code-Graph-RAG uses a three-pass pipeline where Tree-sitter parses source files into ASTs, extracts definition spans and call sites, and streams the resulting nodes and edges into Memgraph via a generic ingestor interface.**

Code-Graph-RAG is an open-source tool that transforms codebases into queryable graph databases. By leveraging Tree-sitter's language-agnostic parsing capabilities, the system extracts syntactic facts—such as function definitions, class declarations, and call relationships—and persists them as nodes and edges in Memgraph. This article breaks down exactly how the Tree-sitter integration works, from parser initialization to batched Cypher ingestion.

## The Three-Pass Pipeline Architecture

The core ingestion logic lives in [`codebase_rag/graph_updater.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/graph_updater.py), where the `GraphUpdater` class orchestrates a three-pass analysis. This design separates structural bookkeeping from syntactic analysis, enabling efficient incremental updates.

### Pass 1 – Structure Identification

The `StructureProcessor` performs a language-agnostic walk of the repository's file tree. This pass records the shape of each module—packages, directories, and file organization—without invoking Tree-sitter. It establishes the foundational context required for later passes to resolve qualified names and file-level metadata.

### Pass 2 – Definition Extraction

The `DefinitionProcessor` drives Tree-sitter parsing for every supported file. According to the source code in [`codebase_rag/graph_updater.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/graph_updater.py), the system initializes language-specific `Parser` objects from compiled Tree-sitter grammars mapped in [`codebase_rag/constants/languages.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/constants/languages.py) [1][2].

For each file, the processor executes queries defined in [`codebase_rag/parsers/language_spec.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/parsers/language_spec.py) to select relevant syntax nodes—such as `function_declaration`, `class_declaration`, and `import_statement`. As it walks the Tree-sitter AST, it builds **definition spans** (e.g., `CppDefinitionSpan`, `FunctionLocation`) that capture the qualified name, node label (FUNCTION, CLASS, MODULE), and exact source range.

These spans are indexed in an in-memory **FunctionRegistryTrie** and cached to disk via a **BoundedASTCache**. The extracted definitions are then emitted to Memgraph as nodes of type `cs.NodeLabel` through the `_sink.ensure_node_batch` batching interface [2].

### Pass 3 – Call-Site Resolution

After the registry is populated with definitions, the `CallProcessor` parses files a second time using Tree-sitter to locate call expressions. It cross-references each call site against the **FunctionRegistryTrie** to resolve callees, including cross-file and macro-expanded targets. The processor creates `CALLS` edges in Memgraph via `_sink.ensure_relationship_batch`, completing the graph structure by linking callers to callees.

## Tree-sitter Integration and Language Support

Code-Graph-RAG abstracts Tree-sitter complexities behind a processor factory pattern. The system loads compiled Tree-sitter grammars into Python `Parser` objects, then binds them to file extensions via the configuration in [`codebase_rag/constants/languages.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/constants/languages.py).

### Language-Specific Queries

The [`language_spec.py`](https://github.com/vitali87/code-graph-rag/blob/main/language_spec.py) module houses Tree-sitter query strings tailored to each supported language. These queries target specific AST node types to extract:
- Function and method declarations
- Class and struct definitions
- Import and include statements

This query-driven approach makes the parser extensible; adding a new language requires only a compiled grammar and a corresponding query specification.

### Hybrid Front-Ends for C/C++

For languages where Tree-sitter cannot provide macro-expansion information (notably C and C++), Code-Graph-RAG optionally runs a *libclang* frontend. As implemented in [`codebase_rag/graph_updater.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/graph_updater.py) [3], the hybrid mode still relies on Tree-sitter definition spans for the caller side, joining macro uses to the tightest containing Tree-sitter node. This combination captures preprocessor semantics that pure Tree-sitter parsing would miss.

## Incremental Parsing and Caching

To avoid re-parsing unchanged files, the system implements incremental update logic. After each run, `GraphUpdater` writes a **parser fingerprint** and file-hash caches. On subsequent runs, the graph-rehydration step loads previously stored definition spans from the `BoundedASTCache`, allowing Tree-sitter to process only modified files.

The [`codebase_rag/services/resource_cleanup.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/services/resource_cleanup.py) module prunes orphan nodes after incremental runs, ensuring the Memgraph database remains consistent with the current state of the codebase.

## Memgraph Ingestion via the Ingestor Protocol

All node and relationship batches flow through the `FilteringIngestor` wrapper, which implements the `IngestorProtocol`. This layer respects user-defined **captures**—configuration flags that disable specific relationship types or node labels—before persistence.

The final flush occurs via `ingestor.flush_all()`, which pushes batched Cypher `CREATE` statements to the Memgraph instance. This batching minimizes network round-trips and maximizes throughput during large repository ingestion.

```python

# Example: indexing a local repository into Memgraph

from cgr import GraphUpdater
from codebase_rag.services import MemgraphIngestor
from codebase_rag.parsers.factory import ProcessorFactory
from codebase_rag.parsers.frontends import FRONTENDS
from tree_sitter import Parser

# 1️⃣ Build the Memgraph ingestor

ingestor = MemgraphIngestor(
    uri="bolt://localhost:7687", 
    user="memgraph", 
    password="******"
)

# 2️⃣ Load Tree-sitter parsers for required languages

parsers = {
    cs.SupportedLanguage.PYTHON: Parser(),  # from compiled tree-sitter-python

    # … add other languages …

}

# 3️⃣ Create the GraphUpdater

updater = GraphUpdater(
    ingestor=ingestor,
    repo_path=Path("/path/to/your/project"),
    parsers=parsers,
    queries=FRONTENDS,      # language-specific query bundles

    capture=None,           # use default capture (no filters)

)

# 4️⃣ Run a full ingest (force=True forces fresh parsing)

updater.run(force=True)

```

```cypher
-- Sample queries to validate the ingested graph in Memgraph

-- List all functions defined in the project
MATCH (f:Function) 
RETURN f.qualified_name AS function_name;

-- Show call edges (who calls whom)
MATCH (caller)-[:CALLS]->(callee) 
RETURN caller.qualified_name, callee.qualified_name;

```

## Summary

- **Three-pass architecture**: `StructureProcessor` maps the file tree, `DefinitionProcessor` extracts Tree-sitter spans into a registry, and `CallProcessor` resolves call sites into edges.
- **Registry-based resolution**: The `FunctionRegistryTrie` and `BoundedASTCache` enable incremental parsing by caching definition spans between runs.
- **Hybrid parsing**: Tree-sitter handles the bulk of languages, with libclang augmentation for C/C++ macro expansion.
- **Batched ingestion**: The `FilteringIngestor` streams nodes and relationships into Memgraph via efficient Cypher batch operations.

## Frequently Asked Questions

### How does Code-Graph-RAG handle incremental updates?

After each ingestion run, the system writes a parser fingerprint and file-hash cache to disk. On subsequent runs, `GraphUpdater` rehydrates the graph state from the `BoundedASTCache`, skipping files whose hashes match the previous run. Tree-sitter only parses changed files, and [`resource_cleanup.py`](https://github.com/vitali87/code-graph-rag/blob/main/resource_cleanup.py) removes orphaned nodes from Memgraph.

### What is the role of the FunctionRegistryTrie in the parsing pipeline?

The `FunctionRegistryTrie` serves as the central index for all extracted definition spans during Pass 2. During Pass 3, the `CallProcessor` queries this trie to resolve call-site targets to their qualified definitions, enabling the creation of accurate `CALLS` relationships even for cross-file invocations.

### How does the system handle C++ macros that Tree-sitter cannot expand?

For C and C++, Code-Graph-RAG activates a hybrid mode using libclang to capture macro-expanded code. According to [`codebase_rag/graph_updater.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/graph_updater.py) [3], the system joins the macro locations from libclang to the tightest containing Tree-sitter nodes, ensuring that definition spans remain anchored to the original source while capturing preprocessor semantics.

### Can I filter which relationship types are ingested into Memgraph?

Yes. The `FilteringIngestor` wrapper respects user-defined **captures** passed to `GraphUpdater`. By specifying a capture configuration, you can disable specific relationship types (such as `CALLS` or `IMPORTS`) or node labels before they are flushed to the database via `ingestor.flush_all()`.