How Code-Graph-RAG's Tree-sitter Parser Extracts Code Structure into Memgraph

Code-Graph-RAG uses a three-pass pipeline where Tree-sitter parses source files into ASTs, extracts definition spans and call sites, and streams the resulting nodes and edges into Memgraph via a generic ingestor interface.

Code-Graph-RAG is an open-source tool that transforms codebases into queryable graph databases. By leveraging Tree-sitter's language-agnostic parsing capabilities, the system extracts syntactic facts—such as function definitions, class declarations, and call relationships—and persists them as nodes and edges in Memgraph. This article breaks down exactly how the Tree-sitter integration works, from parser initialization to batched Cypher ingestion.

The Three-Pass Pipeline Architecture

The core ingestion logic lives in codebase_rag/graph_updater.py, where the GraphUpdater class orchestrates a three-pass analysis. This design separates structural bookkeeping from syntactic analysis, enabling efficient incremental updates.

Pass 1 – Structure Identification

The StructureProcessor performs a language-agnostic walk of the repository's file tree. This pass records the shape of each module—packages, directories, and file organization—without invoking Tree-sitter. It establishes the foundational context required for later passes to resolve qualified names and file-level metadata.

Pass 2 – Definition Extraction

The DefinitionProcessor drives Tree-sitter parsing for every supported file. According to the source code in codebase_rag/graph_updater.py, the system initializes language-specific Parser objects from compiled Tree-sitter grammars mapped in codebase_rag/constants/languages.py [1][2].

For each file, the processor executes queries defined in codebase_rag/parsers/language_spec.py to select relevant syntax nodes—such as function_declaration, class_declaration, and import_statement. As it walks the Tree-sitter AST, it builds definition spans (e.g., CppDefinitionSpan, FunctionLocation) that capture the qualified name, node label (FUNCTION, CLASS, MODULE), and exact source range.

These spans are indexed in an in-memory FunctionRegistryTrie and cached to disk via a BoundedASTCache. The extracted definitions are then emitted to Memgraph as nodes of type cs.NodeLabel through the _sink.ensure_node_batch batching interface [2].

Pass 3 – Call-Site Resolution

After the registry is populated with definitions, the CallProcessor parses files a second time using Tree-sitter to locate call expressions. It cross-references each call site against the FunctionRegistryTrie to resolve callees, including cross-file and macro-expanded targets. The processor creates CALLS edges in Memgraph via _sink.ensure_relationship_batch, completing the graph structure by linking callers to callees.

Tree-sitter Integration and Language Support

Code-Graph-RAG abstracts Tree-sitter complexities behind a processor factory pattern. The system loads compiled Tree-sitter grammars into Python Parser objects, then binds them to file extensions via the configuration in codebase_rag/constants/languages.py.

Language-Specific Queries

The language_spec.py module houses Tree-sitter query strings tailored to each supported language. These queries target specific AST node types to extract:

• Function and method declarations
• Class and struct definitions
• Import and include statements

This query-driven approach makes the parser extensible; adding a new language requires only a compiled grammar and a corresponding query specification.

Hybrid Front-Ends for C/C++

For languages where Tree-sitter cannot provide macro-expansion information (notably C and C++), Code-Graph-RAG optionally runs a libclang frontend. As implemented in codebase_rag/graph_updater.py [3], the hybrid mode still relies on Tree-sitter definition spans for the caller side, joining macro uses to the tightest containing Tree-sitter node. This combination captures preprocessor semantics that pure Tree-sitter parsing would miss.

Incremental Parsing and Caching

To avoid re-parsing unchanged files, the system implements incremental update logic. After each run, GraphUpdater writes a parser fingerprint and file-hash caches. On subsequent runs, the graph-rehydration step loads previously stored definition spans from the BoundedASTCache, allowing Tree-sitter to process only modified files.

The codebase_rag/services/resource_cleanup.py module prunes orphan nodes after incremental runs, ensuring the Memgraph database remains consistent with the current state of the codebase.

Memgraph Ingestion via the Ingestor Protocol

All node and relationship batches flow through the FilteringIngestor wrapper, which implements the IngestorProtocol. This layer respects user-defined captures—configuration flags that disable specific relationship types or node labels—before persistence.

The final flush occurs via ingestor.flush_all(), which pushes batched Cypher CREATE statements to the Memgraph instance. This batching minimizes network round-trips and maximizes throughput during large repository ingestion.

# Example: indexing a local repository into Memgraph

from cgr import GraphUpdater
from codebase_rag.services import MemgraphIngestor
from codebase_rag.parsers.factory import ProcessorFactory
from codebase_rag.parsers.frontends import FRONTENDS
from tree_sitter import Parser

# 1️⃣ Build the Memgraph ingestor

ingestor = MemgraphIngestor(
    uri="bolt://localhost:7687", 
    user="memgraph", 
    password="******"
)

# 2️⃣ Load Tree-sitter parsers for required languages

parsers = {
    cs.SupportedLanguage.PYTHON: Parser(),  # from compiled tree-sitter-python

    # … add other languages …

}

# 3️⃣ Create the GraphUpdater

updater = GraphUpdater(
    ingestor=ingestor,
    repo_path=Path("/path/to/your/project"),
    parsers=parsers,
    queries=FRONTENDS,      # language-specific query bundles

    capture=None,           # use default capture (no filters)

)

# 4️⃣ Run a full ingest (force=True forces fresh parsing)

updater.run(force=True)

-- Sample queries to validate the ingested graph in Memgraph

-- List all functions defined in the project
MATCH (f:Function) 
RETURN f.qualified_name AS function_name;

-- Show call edges (who calls whom)
MATCH (caller)-[:CALLS]->(callee) 
RETURN caller.qualified_name, callee.qualified_name;

Summary

• Three-pass architecture: StructureProcessor maps the file tree, DefinitionProcessor extracts Tree-sitter spans into a registry, and CallProcessor resolves call sites into edges.
• Registry-based resolution: The FunctionRegistryTrie and BoundedASTCache enable incremental parsing by caching definition spans between runs.
• Hybrid parsing: Tree-sitter handles the bulk of languages, with libclang augmentation for C/C++ macro expansion.
• Batched ingestion: The FilteringIngestor streams nodes and relationships into Memgraph via efficient Cypher batch operations.

Frequently Asked Questions

How does Code-Graph-RAG handle incremental updates?

After each ingestion run, the system writes a parser fingerprint and file-hash cache to disk. On subsequent runs, GraphUpdater rehydrates the graph state from the BoundedASTCache, skipping files whose hashes match the previous run. Tree-sitter only parses changed files, and resource_cleanup.py removes orphaned nodes from Memgraph.

What is the role of the FunctionRegistryTrie in the parsing pipeline?

The FunctionRegistryTrie serves as the central index for all extracted definition spans during Pass 2. During Pass 3, the CallProcessor queries this trie to resolve call-site targets to their qualified definitions, enabling the creation of accurate CALLS relationships even for cross-file invocations.

How does the system handle C++ macros that Tree-sitter cannot expand?

For C and C++, Code-Graph-RAG activates a hybrid mode using libclang to capture macro-expanded code. According to codebase_rag/graph_updater.py [3], the system joins the macro locations from libclang to the tightest containing Tree-sitter nodes, ensuring that definition spans remain anchored to the original source while capturing preprocessor semantics.

Can I filter which relationship types are ingested into Memgraph?

Yes. The FilteringIngestor wrapper respects user-defined captures passed to GraphUpdater. By specifying a capture configuration, you can disable specific relationship types (such as CALLS or IMPORTS) or node labels before they are flushed to the database via ingestor.flush_all().

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}