# How the Code Graph Is Generated in code-graph-rag: A Deep Dive Into the Multi-Pass Pipeline

> Discover how code-graph-rag generates its code graph via a language-agnostic multi-pass pipeline. Learn about parsing, relationship extraction, and ingestion into Neo4j.

- Repository: [Vitali Avagyan/code-graph-rag](https://github.com/vitali87/code-graph-rag)
- Tags: deep-dive
- Published: 2026-08-14

---

**The code graph in code-graph-rag is generated through a language-agnostic, multi-pass analysis pipeline that parses source files, extracts definitions and relationships, and streams nodes and edges to a configurable ingestor such as Neo4j.**

The repository *vitali87/code-graph-rag* transforms any codebase into a queryable graph structure suitable for retrieval-augmented generation. At its core, the system orchestrates lazy-loaded parsers, structured analysis passes, and deferred relationship resolution. This article explains exactly how the pipeline works, referencing the actual source implementation.

## Loading Language Parsers Lazily

The pipeline begins in [`codebase_rag/parser_loader.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/parser_loader.py) with `load_parsers()`. This function creates **tree-sitter** `Parser` objects and compiled queries for each supported language without paying upfront costs for grammars you don't need.

```python
from codebase_rag.parser_loader import load_parsers
parsers, queries = load_parsers()

```

The implementation uses `_loader_cache` to store grammar modules per language. A custom `_LazyGrammarStore` defers loading until first access. For each language, `_create_language_queries()` assembles a `LanguageQueries` object containing optional queries for functions, classes, calls, imports, locals, and highlights.

Grammar resolution happens via `_get_language_library()`. This helper either imports a pre-built submodule or falls back to building bindings on-the-fly through `subprocess.run([sys.executable, "setup.py", "build_ext", "--inplace"])`.

> Source: [[`parser_loader.py`](https://github.com/vitali87/code-graph-rag/blob/main/parser_loader.py)](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/parser_loader.py#L1-L88)

## Instantiating the GraphUpdater

The central orchestrator is `GraphUpdater` in [`codebase_rag/graph_updater.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/graph_updater.py). It accepts four key dependencies:

```python
from codebase_rag.graph_updater import GraphUpdater

updater = GraphUpdater(
    ingestor=my_ingestor,
    repo_path=Path("/path/to/repo"),
    parsers=parsers,
    queries=queries,
)

```

- **ingestor**: Any implementation of **IngestorProtocol** (Neo4j, in-memory, etc.)
- **repo_path**: Root directory of the target codebase
- **parsers**: Lazy-loaded tree-sitter parsers from `load_parsers()`
- **queries**: Per-language query objects for syntax extraction

> Source: [[`graph_updater.py`](https://github.com/vitali87/code-graph-rag/blob/main/graph_updater.py)](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/graph_updater.py#L13-L33)

## The Multi-Pass Ingestion Pipeline

The `run()` method executes a carefully ordered sequence of passes. Each pass builds upon the previous, accumulating knowledge about the codebase structure.

### Pass 1: Structure Detection

**`self.factory.structure_processor.identify_structure()`** establishes project-wide organization—modules, packages, and directory relationships. This provides the foundation for qualified name resolution later.

### Pass 2: File Parsing and Definition Extraction

**`_process_files()`** iterates every source file, selects the appropriate parser, and emits:

- **Nodes**: Functions, classes, modules (definitions)
- **Edges**: Import relationships between modules

Tree-sitter queries extract concrete syntax nodes based on the `LanguageQueries` created earlier.

### Pass 3: Call Site Resolution

**`_process_function_calls()`** builds a **name-trie** (`FunctionRegistryTrie`) to resolve call sites. It creates `CALLS` edges linking caller functions to their callees using qualified name matching.

### Hybrid Frontends for Enhanced Semantics

Before or after Pass 2, language-specific frontends inject richer facts:

| Frontend | Method | Capabilities |
|----------|--------|--------------|
| **C++** | `_run_cpp_frontend()` | Macro expansion, template instantiation via libclang |
| **C#** | `_run_csharp_frontend()` | LINQ query analysis, nullable reference tracking via Roslyn |
| **Go** | `_run_go_frontend()` | Interface satisfaction, type inference via go/types |

These hybrid frontends complement tree-sitter by providing compiler-accurate semantic information that pure syntactic analysis cannot capture.

### Deferred Relationship Resolution

After main passes complete, **deferred joins** resolve relationships that require global knowledge:

- `_join_csharp_partials()` — merges partial class definitions
- `_join_go_implements()` — links interfaces to implementing types
- `_resolve_hybrid_macro_calls()` — connects macro expansions to original sites

This deferred approach ensures that inheritance, implementation, and macro relationships resolve correctly once all definition spans are known.

### Endpoint Detection

**`_emit_pending_endpoints()`** and **`_emit_route_call_endpoints()`** scan for web framework patterns (Express, net/http, Gin, Echo, etc.) and emit `EXPOSES` edges connecting HTTP routes to their handler functions.

### Optional Static Analysis

**`self.finding_analyzer.analyze(...)`** runs ast-grep based rules for security issues and code smells, emitting findings as additional nodes if enabled.

### Final Persistence

**`self.ingestor.flush_all()`** commits all batched nodes and edges to the backend database.

> Source: [[`graph_updater.py`](https://github.com/vitali87/code-graph-rag/blob/main/graph_updater.py)](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/graph_updater.py#L9-L15)

## Emitting Nodes and Edges Through Filtering

All emission flows through a **filtering wrapper** (`FilteringIngestor`) stored in `self._sink`. This wrapper respects capture configuration—allowing users to disable specific relationship types—before forwarding to the raw ingestor.

Typical emission for a function call:

```python
self._sink.ensure_relationship_batch(
    (caller_label, cs.KEY_QUALIFIED_NAME, caller_qn),
    cs.RelationshipType.CALLS,
    (cs.NodeLabel.FUNCTION, cs.KEY_QUALIFIED_NAME, callee_qn),
)

```

The batched API minimizes database round-trips during large ingestion runs.

> Source: [[`graph_updater.py`](https://github.com/vitali87/code-graph-rag/blob/main/graph_updater.py)](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/graph_updater.py#L1249-L1255)

## Incremental vs. Full Builds

The pipeline supports two modes controlled by the `force` parameter:

- **`force=True`** — Full build: clears caches, parses every file, guarantees fresh graph
- **Incremental** — Skips unchanged files by comparing **file hashes** (`_hash_file`) and directory modification times

Early termination occurs when `_is_already_in_sync()` detects no changes since the last run. Hash persistence uses `_load_hash_cache` and `_save_hash_cache`.

> Source: Hash handling in [`graph_updater.py`](https://github.com/vitali87/code-graph-rag/blob/main/graph_updater.py) — functions `_hash_file`, `_load_hash_cache`, `_save_hash_cache` (lines 27-63)

## Complete Usage Example

The CLI and example script demonstrate end-to-end execution:

```bash

# CLI invocation

python -m codebase_rag.cli --repo /path/to/codebase --ingest neo4j

```

Programmatic usage:

```python
from pathlib import Path
from codebase_rag.parser_loader import load_parsers
from codebase_rag.ingestors import InMemoryIngestor
from codebase_rag.graph_updater import GraphUpdater

# 1. Load parsers (once per process)

parsers, queries = load_parsers()

# 2. Create ingestor

ingestor = InMemoryIngestor()

# 3. Build the updater

updater = GraphUpdater(
    ingestor=ingestor,
    repo_path=Path("/my/project"),
    parsers=parsers,
    queries=queries,
)

# 4. Run ingestion

updater.run(force=True)   # full build

# updater.run()           # incremental: skips unchanged files

```

> Source: [[`examples/graph_export_example.py`](https://github.com/vitali87/code-graph-rag/blob/main/examples/graph_export_example.py)](https://github.com/vitali87/code-graph-rag/blob/main/examples/graph_export_example.py)

## Key Source Files

| File | Responsibility |
|------|----------------|
| [`codebase_rag/graph_updater.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/graph_updater.py) | Orchestrates multi-pass parsing, deferred joins, graph emission |
| [`codebase_rag/parser_loader.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/parser_loader.py) | Lazy tree-sitter grammar loading, query compilation |
| [`codebase_rag/main.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/main.py) | High-level entry point for CLI and library use |
| [`codebase_rag/cli.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/cli.py) | Command-line interface wiring options to `GraphUpdater` |
| [`examples/graph_export_example.py`](https://github.com/vitali87/code-graph-rag/blob/main/examples/graph_export_example.py) | Minimal demonstration of pipeline invocation |

## Summary

The code graph generation in code-graph-rag follows a rigorous pipeline:

- **Lazy loading** of tree-sitter grammars minimizes startup overhead
- **Multi-pass analysis** progressively builds structure, definitions, calls, and semantic facts
- **Hybrid frontends** augment syntactic parsing with compiler-grade insights for C++, C#, and Go
- **Deferred resolution** ensures global relationships (inheritance, implements, macros) resolve correctly
- **Filtering ingestor** enables fine-grained control over captured relationships
- **Incremental builds** skip unchanged files for fast iteration

The result is a dense, language-agnostic code graph ready for RAG queries, security analysis, and architectural exploration.

## Frequently Asked Questions

### What parsing technology does code-graph-rag use for code graph generation?

The system primarily uses **tree-sitter** for language-agnostic syntactic analysis. Tree-sitter provides incremental, error-resilient parsing across many languages. For enhanced accuracy, hybrid frontends integrate **libclang** (C++), **Roslyn** (C#), and **go/types** (Go) to capture semantic facts that pure syntax cannot express.

### How does the pipeline handle large codebases efficiently?

Three mechanisms optimize performance: **lazy grammar loading** avoids importing unused parsers; **incremental builds** skip unchanged files by comparing SHA-256 hashes; and **batched emission** reduces database round-trips. The early-exit check `_is_already_in_sync()` aborts immediately when no changes are detected.

### Can I customize which relationships appear in the generated graph?

Yes. The `FilteringIngestor` wrapper respects capture configuration, allowing you to disable specific relationship types (e.g., exclude `CALLS` edges while keeping `CONTAINS`). Pass a custom configuration to `GraphUpdater` or use CLI flags to control the emitted graph structure.

### What web frameworks does the endpoint detection support?

The `_emit_pending_endpoints()` and `_emit_route_call_endpoints()` methods recognize route registrations from **Express** (Node.js), **net/http** and **Gin** (Go), **Echo**, **Flask** patterns, and other common frameworks. The `EXPOSES` edges link HTTP paths directly to their handler functions for API analysis and documentation generation.