# How Code-Graph-RAG Uses Tree-sitter for Multi-Language Code Analysis

> Explore how Code-Graph-RAG utilizes Tree-sitter for efficient, multi-language code analysis. Discover its power in semantic extraction and incremental code-graph construction.

- Repository: [Vitali Avagyan/code-graph-rag](https://github.com/vitali87/code-graph-rag)
- Tags: how-to-guide
- Published: 2026-08-11

---

**Code-Graph-RAG leverages Tree-sitter to parse Python, JavaScript, TypeScript, Rust, Java, and other languages into abstract syntax trees (ASTs), enabling query-driven semantic extraction and incremental code-graph construction.**

Code-Graph-RAG is an open-source retrieval-augmented generation (RAG) framework that builds rich code graphs from diverse programming languages. At its core, the system uses **Tree-sitter**—GitHub's parsing library—to create language-agnostic ASTs that power cross-language analysis, type inference, and call-graph resolution.

## Grammar Discovery and Submodule Management

The framework dynamically discovers language grammars through Git submodules. When you run `cgr init` or `cgr add-language`, the tool executes `_add_git_submodule` in [`codebase_rag/tools/language.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/tools/language.py) to pull the appropriate Tree-sitter grammar repository.

After downloading, the `_parse_tree_sitter_json` function (lines 31-55 in [`codebase_rag/tools/language.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/tools/language.py)) reads the grammar's [`grammar.json`](https://github.com/vitali87/code-graph-rag/blob/main/grammar.json) file to extract the canonical language name and supported file extensions. This automated discovery eliminates hard-coded language lists and allows the system to support new languages by simply adding their Tree-sitter grammar as a submodule.

```bash

# Add a new language grammar as a submodule

cgr add-language --url https://github.com/tree-sitter/tree-sitter-go --name go

```

## Lazy Parser Initialization and AST Caching

Code-Graph-RAG creates **Tree-sitter Parser instances lazily** and caches them globally to avoid the overhead of repeated initialization. When indexing a Python file for the first time, the system loads the compiled grammar and constructs a `Language` object from `tree_sitter_python.language()`.

The `ASTCacheProtocol` implementation in [`codebase_rag/ast_cache.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/ast_cache.py) stores parsed trees keyed by file path. This cache enables **incremental re-indexing** and prevents redundant parsing when the same file is visited from multiple call-graph paths. The initial imports and cache setup appear in [`codebase_rag/parsers/py/ast_analyzer.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/parsers/py/ast_analyzer.py) (lines 1-14).

```python
from tree_sitter import Language, Parser
import tree_sitter_python

# Build Language from compiled grammar and create parser

py_lang = Language(tree_sitter_python.language())
parser = Parser(py_lang)

# Parse source and cache the tree

source_bytes = Path("example.py").read_bytes()
tree = parser.parse(source_bytes)

```

## Query-Driven AST Traversal

The core analysis logic resides in `PythonAstAnalyzerMixin` (lines 54-92 in [`codebase_rag/parsers/py/ast_analyzer.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/parsers/py/ast_analyzer.py)). This mixin uses **Tree-sitter Query objects** to capture semantic nodes like assignments, function definitions, and import statements.

The workflow follows three steps:

1. **Query Compilation**: Language-specific query strings (defined in [`codebase_rag/constants/ast_python.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/constants/ast_python.py)) are compiled once and cached via `get_cached_query`.
2. **Cursor Execution**: A `QueryCursor` iterates over the AST root node to find capture groups (e.g., `@assignment`, `@for_stmt`).
3. **Node Processing**: Helper methods like `_process_assignment_simple` and `_process_assignment_complex` extract type information from captured nodes, building a local variable type map for accurate call-graph construction.

If a query fails due to missing grammars, the system falls back to manual stack-based traversal.

```python
from codebase_rag.parsers.py.ast_analyzer import _PY_TRAVERSE_QUERY, get_cached_query

# Obtain cached query and create cursor

query = get_cached_query(py_lang, _PY_TRAVERSE_QUERY)
cursor = query.Cursor()

# Capture semantic nodes from the AST root

captures = cursor.captures(tree.root_node)
assignments = captures.get("assignment", [])

for node in assignments:
    print(f"Assignment found at line {node.start_point[0]}")

```

## Cross-Language Semantic Resolution

Code-Graph-RAG provides a unified interface for cross-language analysis through shared utilities. The `Node` and `Query` abstractions from Tree-sitter are reused across language parsers, as seen in [`codebase_rag/parsers/js_ts/utils.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/parsers/js_ts/utils.py) (lines 4-7).

The **fully-qualified name (FQN) resolver** in [`codebase_rag/utils/fqn_resolver.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/utils/fqn_resolver.py) (lines 12-20) walks cached ASTs using Tree-sitter `Node` objects to locate exact definitions across language boundaries. This enables the system to resolve method calls in JavaScript that reference Python backends, or track Rust functions called from TypeScript bindings.

Language-specific query patterns are centralized in `codebase_rag/constants/` (e.g., [`ast_python.py`](https://github.com/vitali87/code-graph-rag/blob/main/ast_python.py), [`ast_rust.py`](https://github.com/vitali87/code-graph-rag/blob/main/ast_rust.py)), ensuring consistent capture group naming across different grammars.

```python
from codebase_rag.utils.fqn_resolver import resolve_fqn

# Resolve a symbol across cached ASTs

node = resolve_fqn("my_pkg.module.MyClass.method", ast_cache, parser_cache)
print(node.type)  # "function_definition" or "method_declaration"

```

## Summary

- **Grammar Discovery**: The `_parse_tree_sitter_json` function in [`codebase_rag/tools/language.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/tools/language.py) automates language support by parsing Tree-sitter grammar metadata from Git submodules.
- **Performance Optimization**: Parser instances and AST trees are cached globally via `ASTCacheProtocol` to eliminate redundant parsing during incremental indexing.
- **Semantic Extraction**: `PythonAstAnalyzerMixin` uses compiled Tree-sitter queries and `QueryCursor` captures to extract functions, classes, and type information from AST nodes.
- **Cross-Language Support**: Shared utilities like [`fqn_resolver.py`](https://github.com/vitali87/code-graph-rag/blob/main/fqn_resolver.py) and constants files provide a unified interface for multi-language code-graph construction.

## Frequently Asked Questions

### What programming languages does Code-Graph-RAG support through Tree-sitter?

The framework supports **Python, JavaScript, TypeScript, Rust, Java, C/C++, Go, Lua, PHP, and C#**, with the ability to add new languages by registering their Tree-sitter grammars as Git submodules. The `_parse_tree_sitter_json` function automatically detects file extensions and language names from each grammar's metadata.

### How does Code-Graph-RAG cache Tree-sitter parsers for performance?

The system implements **lazy initialization** combined with the `ASTCacheProtocol`. Parser objects are created once per language and reused across the codebase, while parsed ASTs are stored in an in-memory cache keyed by file path. This architecture prevents re-parsing files during incremental updates or repeated cross-reference lookups.

### What happens when a Tree-sitter query fails or encounters a missing grammar?

When a query compilation fails or a grammar is unavailable, Code-Graph-RAG falls back to **manual stack-based AST traversal**. This ensures robustness while maintaining the efficiency of query-driven capture groups for supported languages. The fallback logic is integrated into the mixin classes like `PythonAstAnalyzerMixin`.

### How does the system resolve symbols across different programming languages?

The `resolve_fqn` utility in [`codebase_rag/utils/fqn_resolver.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/utils/fqn_resolver.py) traverses cached ASTs using Tree-sitter `Node` objects to locate symbol definitions. By using language-agnostic Node APIs and standardized query patterns from `constants/ast_*.py` files, the resolver can trace calls from TypeScript to Python or Rust, enabling true cross-language code-graph analysis.