How the Code Graph Is Generated in code-graph-rag: A Deep Dive Into the Multi-Pass Pipeline

The code graph in code-graph-rag is generated through a language-agnostic, multi-pass analysis pipeline that parses source files, extracts definitions and relationships, and streams nodes and edges to a configurable ingestor such as Neo4j.

The repository vitali87/code-graph-rag transforms any codebase into a queryable graph structure suitable for retrieval-augmented generation. At its core, the system orchestrates lazy-loaded parsers, structured analysis passes, and deferred relationship resolution. This article explains exactly how the pipeline works, referencing the actual source implementation.

Loading Language Parsers Lazily

The pipeline begins in codebase_rag/parser_loader.py with load_parsers(). This function creates tree-sitter Parser objects and compiled queries for each supported language without paying upfront costs for grammars you don't need.

from codebase_rag.parser_loader import load_parsers
parsers, queries = load_parsers()

The implementation uses _loader_cache to store grammar modules per language. A custom _LazyGrammarStore defers loading until first access. For each language, _create_language_queries() assembles a LanguageQueries object containing optional queries for functions, classes, calls, imports, locals, and highlights.

Grammar resolution happens via _get_language_library(). This helper either imports a pre-built submodule or falls back to building bindings on-the-fly through subprocess.run([sys.executable, "setup.py", "build_ext", "--inplace"]).

Source: [parser_loader.py](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/parser_loader.py#L1-L88)

Instantiating the GraphUpdater

The central orchestrator is GraphUpdater in codebase_rag/graph_updater.py. It accepts four key dependencies:

from codebase_rag.graph_updater import GraphUpdater

updater = GraphUpdater(
    ingestor=my_ingestor,
    repo_path=Path("/path/to/repo"),
    parsers=parsers,
    queries=queries,
)

• ingestor: Any implementation of IngestorProtocol (Neo4j, in-memory, etc.)
• repo_path: Root directory of the target codebase
• parsers: Lazy-loaded tree-sitter parsers from load_parsers()
• queries: Per-language query objects for syntax extraction

Source: [graph_updater.py](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/graph_updater.py#L13-L33)

The Multi-Pass Ingestion Pipeline

The run() method executes a carefully ordered sequence of passes. Each pass builds upon the previous, accumulating knowledge about the codebase structure.

Pass 1: Structure Detection

self.factory.structure_processor.identify_structure() establishes project-wide organization—modules, packages, and directory relationships. This provides the foundation for qualified name resolution later.

Pass 2: File Parsing and Definition Extraction

_process_files() iterates every source file, selects the appropriate parser, and emits:

• Nodes: Functions, classes, modules (definitions)
• Edges: Import relationships between modules

Tree-sitter queries extract concrete syntax nodes based on the LanguageQueries created earlier.

Pass 3: Call Site Resolution

_process_function_calls() builds a name-trie (FunctionRegistryTrie) to resolve call sites. It creates CALLS edges linking caller functions to their callees using qualified name matching.

Hybrid Frontends for Enhanced Semantics

Before or after Pass 2, language-specific frontends inject richer facts:

Frontend	Method	Capabilities
C++	_run_cpp_frontend()	Macro expansion, template instantiation via libclang
C#	_run_csharp_frontend()	LINQ query analysis, nullable reference tracking via Roslyn
Go	_run_go_frontend()	Interface satisfaction, type inference via go/types

These hybrid frontends complement tree-sitter by providing compiler-accurate semantic information that pure syntactic analysis cannot capture.

Deferred Relationship Resolution

After main passes complete, deferred joins resolve relationships that require global knowledge:

• _join_csharp_partials() — merges partial class definitions
• _join_go_implements() — links interfaces to implementing types
• _resolve_hybrid_macro_calls() — connects macro expansions to original sites

This deferred approach ensures that inheritance, implementation, and macro relationships resolve correctly once all definition spans are known.

Endpoint Detection

_emit_pending_endpoints() and _emit_route_call_endpoints() scan for web framework patterns (Express, net/http, Gin, Echo, etc.) and emit EXPOSES edges connecting HTTP routes to their handler functions.

Optional Static Analysis

self.finding_analyzer.analyze(...) runs ast-grep based rules for security issues and code smells, emitting findings as additional nodes if enabled.

Final Persistence

self.ingestor.flush_all() commits all batched nodes and edges to the backend database.

Source: [graph_updater.py](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/graph_updater.py#L9-L15)

Emitting Nodes and Edges Through Filtering

All emission flows through a filtering wrapper (FilteringIngestor) stored in self._sink. This wrapper respects capture configuration—allowing users to disable specific relationship types—before forwarding to the raw ingestor.

Typical emission for a function call:

self._sink.ensure_relationship_batch(
    (caller_label, cs.KEY_QUALIFIED_NAME, caller_qn),
    cs.RelationshipType.CALLS,
    (cs.NodeLabel.FUNCTION, cs.KEY_QUALIFIED_NAME, callee_qn),
)

The batched API minimizes database round-trips during large ingestion runs.

Source: [graph_updater.py](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/graph_updater.py#L1249-L1255)

Incremental vs. Full Builds

The pipeline supports two modes controlled by the force parameter:

• force=True — Full build: clears caches, parses every file, guarantees fresh graph
• Incremental — Skips unchanged files by comparing file hashes (_hash_file) and directory modification times

Early termination occurs when _is_already_in_sync() detects no changes since the last run. Hash persistence uses _load_hash_cache and _save_hash_cache.

Source: Hash handling in graph_updater.py — functions _hash_file, _load_hash_cache, _save_hash_cache (lines 27-63)

Complete Usage Example

The CLI and example script demonstrate end-to-end execution:

# CLI invocation

python -m codebase_rag.cli --repo /path/to/codebase --ingest neo4j

Programmatic usage:

from pathlib import Path
from codebase_rag.parser_loader import load_parsers
from codebase_rag.ingestors import InMemoryIngestor
from codebase_rag.graph_updater import GraphUpdater

# 1. Load parsers (once per process)

parsers, queries = load_parsers()

# 2. Create ingestor

ingestor = InMemoryIngestor()

# 3. Build the updater

updater = GraphUpdater(
    ingestor=ingestor,
    repo_path=Path("/my/project"),
    parsers=parsers,
    queries=queries,
)

# 4. Run ingestion

updater.run(force=True)   # full build

# updater.run()           # incremental: skips unchanged files

Source: [examples/graph_export_example.py](https://github.com/vitali87/code-graph-rag/blob/main/examples/graph_export_example.py)

Key Source Files

File	Responsibility
codebase_rag/graph_updater.py	Orchestrates multi-pass parsing, deferred joins, graph emission
codebase_rag/parser_loader.py	Lazy tree-sitter grammar loading, query compilation
codebase_rag/main.py	High-level entry point for CLI and library use
codebase_rag/cli.py	Command-line interface wiring options to GraphUpdater
examples/graph_export_example.py	Minimal demonstration of pipeline invocation

Summary

The code graph generation in code-graph-rag follows a rigorous pipeline:

• Lazy loading of tree-sitter grammars minimizes startup overhead
• Multi-pass analysis progressively builds structure, definitions, calls, and semantic facts
• Hybrid frontends augment syntactic parsing with compiler-grade insights for C++, C#, and Go
• Deferred resolution ensures global relationships (inheritance, implements, macros) resolve correctly
• Filtering ingestor enables fine-grained control over captured relationships
• Incremental builds skip unchanged files for fast iteration

The result is a dense, language-agnostic code graph ready for RAG queries, security analysis, and architectural exploration.

Frequently Asked Questions

What parsing technology does code-graph-rag use for code graph generation?

The system primarily uses tree-sitter for language-agnostic syntactic analysis. Tree-sitter provides incremental, error-resilient parsing across many languages. For enhanced accuracy, hybrid frontends integrate libclang (C++), Roslyn (C#), and go/types (Go) to capture semantic facts that pure syntax cannot express.

How does the pipeline handle large codebases efficiently?

Three mechanisms optimize performance: lazy grammar loading avoids importing unused parsers; incremental builds skip unchanged files by comparing SHA-256 hashes; and batched emission reduces database round-trips. The early-exit check _is_already_in_sync() aborts immediately when no changes are detected.

Can I customize which relationships appear in the generated graph?

Yes. The FilteringIngestor wrapper respects capture configuration, allowing you to disable specific relationship types (e.g., exclude CALLS edges while keeping CONTAINS). Pass a custom configuration to GraphUpdater or use CLI flags to control the emitted graph structure.

What web frameworks does the endpoint detection support?

The _emit_pending_endpoints() and _emit_route_call_endpoints() methods recognize route registrations from Express (Node.js), net/http and Gin (Go), Echo, Flask patterns, and other common frameworks. The EXPOSES edges link HTTP paths directly to their handler functions for API analysis and documentation generation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}