How Code‑Graph‑RAG Handles Large Codebases: Bounded Caching, Batch Ingestion, and Incremental Parsing

Code‑Graph‑RAG uses a bounded LRU AST cache, multi‑pass incremental parsing, batched graph ingestion, and streaming file access to keep memory and latency predictable even in monorepos with tens of thousands of files.

Code‑Graph‑RAG is an open‑source RAG system designed to index, query, and edit very large, polyglot repositories. Its scalability rests on several coordinated components that work together to prevent out‑of‑memory errors and maintain fast query performance. Below is a detailed breakdown of how each mechanism works, grounded in the actual source code from vitali87/code-graph-rag.

Memory‑Bounded AST Caching with BoundedASTCache

The core defense against memory exhaustion is BoundedASTCache in codebase_rag/ast_cache.py. This LRU cache enforces two hard limits:

• CACHE_MAX_ENTRIES: Maximum number of AST objects stored
• CACHE_MAX_MEMORY_MB: Maximum total memory consumed by the cache

When either limit is exceeded, the cache evicts the oldest ASTs. Evicted files are not lost—_load_ast_from_disk() lazily re‑parses them on demand. This guarantees that even a repository with tens of thousands of files never exhausts RAM.

from codebase_rag.ast_cache import BoundedASTCache

cache = BoundedASTCache()

# ASTs are automatically managed; old entries evicted when limits hit

ast = cache.get_or_parse("src/large_module.py", parse_fn)

Incremental Multi‑Pass Parsing

The indexer in codebase_rag/graph_updater.py runs three distinct passes to build a complete code graph:

1. Tree‑sitter parsing – Extract syntax nodes and basic symbols
2. Type‑inference & relationship resolution – Resolve imports, inheritance, and call edges
3. Graph‑ingestion – Write nodes and relationships to Memgraph

All files parsed in the current run are recorded in _parsed_files regardless of cache eviction. This ensures that later passes can re‑parse evicted files without losing cross‑file edges. The result is a robust incremental build system that handles partial cache misses gracefully.

Batched Ingestion into Memgraph and Vector Stores

Writing millions of symbols to a graph database or vector store one‑by‑one would cripple performance. Code‑Graph‑RAG solves this with configurable batching:

Store	Mechanism	Configuration
Memgraph	MEMGRAPH_BATCH_SIZE (default 1000 nodes/edges per batch)	codebase_rag/config.py
Vector store (Qdrant/Milvus)	store_embedding_batch() bulk insertion	codebase_rag/vector_store.py

The settings.resolve_batch_size() helper in codebase_rag/config.py allows users to tune batch sizes for their hardware.

from codegraph_rag.sdk.graph_loader import GraphLoader

loader = GraphLoader(repo_path="/path/to/large/repo")
loader.load()  # Runs three passes with BoundedASTCache

loader.ingest_batch(batch_size=2000)  # Override default MEMGRAPH_BATCH_SIZE

Large‑File Streaming via MCP Tools

The MCP‑based file‑access tools in codebase_rag/tools/tool_descriptions.py accept offset and limit arguments. This allows callers to request only a slice of a huge source file, preventing the entire file from loading into memory when only a portion is needed (e.g., for a diff or partial context).

# Streaming a large file via the MCP API

await mcp_registry.read_file(
    "src/very_large_file.py",
    offset=500_000,   # Start at byte 500k

    limit=10_000      # Read only 10k bytes

)

Configurable Hardware Tuning

All performance‑critical parameters are exposed through codebase_rag/config.py:

• MEMGRAPH_BATCH_SIZE
• CACHE_MAX_ENTRIES
• CACHE_MAX_MEMORY_MB

Users can adjust these for their deployment target—whether a laptop with 16 GB RAM or a server with 256 GB.

Hybrid Front‑Ends for C/C++ and C#

When a repository contains C/C++ or C# files, optional libclang or Roslyn front‑ends can pre‑process macros and semantic facts. These are only enabled when the required toolchain is present; otherwise the pure tree‑sitter pipeline runs. This prevents crashes on huge repositories that lack native tooling.

The relevant methods in codebase_rag/graph_updater.py are:

• _run_cpp_frontend()
• _run_csharp_frontend()

Parallel and Debounced File‑System Watching

In watch mode, realtime_updater.py debounces file‑system events and processes them in configurable batches. This prevents a flood of updates from overwhelming the indexer when many files change at once—common after large refactors or branch switches.

# Watch mode with batched updates

python realtime_updater.py --repo-path /path/to/repo --batch-size 50

Benchmarked Performance

The optimize/profile_io.py benchmark demonstrates Code‑Graph‑RAG ingesting a monorepo with 10,000+ nodes and 20,000+ relationships without OOM. Query latency stays low thanks to indexed graph traversal and vector‑search caching.

Quick Start for Large Codebases

# 1. Start the bundled Memgraph+Qdrant stack (no Docker compose needed)

cgr daemon up

# 2. Index a large repository; BoundedASTCache activates automatically

cgr start --repo-path /path/to/huge-repo --update-graph

# 3. Query with confidence in stable memory use

cgr query "Find all functions that import `numpy`"

Summary

• BoundedASTCache in codebase_rag/ast_cache.py caps memory use with LRU eviction and lazy re‑parsing

• Three‑pass incremental parsing in codebase_rag/graph_updater.py tracks parsed files across cache misses

• Batched ingestion via MEMGRAPH_BATCH_SIZE and store_embedding_batch() minimizes database round‑trips

• Streaming file access via MCP offset/limit parameters avoids loading massive files whole

• Configurable batch sizes in codebase_rag/config.py let users tune for their hardware

• Optional native front‑ends for C/C++ and C# gracefully degrade when toolchains are absent

• Debounced watching in realtime_updater.py protects against update floods

Frequently Asked Questions

What happens when the AST cache is full?

The BoundedASTCache evicts the least‑recently‑used ASTs based on the CACHE_MAX_ENTRIES and CACHE_MAX_MEMORY_MB limits. Evicted files are re‑parsed on demand via _load_ast_from_disk() when later passes need them, so no data is lost—only temporarily unloaded from memory.

Can I tune batch sizes for my hardware?

Yes. Adjust MEMGRAPH_BATCH_SIZE in codebase_rag/config.py or override it programmatically with loader.ingest_batch(batch_size=...). The settings.resolve_batch_size() helper provides sensible defaults while allowing full customization.

How does Code‑Graph‑RAG avoid loading multi‑gigabyte source files?

The MCP tools expose offset and limit parameters in codebase_rag/tools/tool_descriptions.py. Callers request specific byte ranges, so the system never materializes an entire massive file in memory unless explicitly asked.

Does watching a large repository cause re‑indexing storms?

No. The realtime_updater.py watcher debounces events and processes them in batches. The batch_size argument configures how many changes are coalesced before triggering an incremental update, preventing CPU and I/O spikes during bulk file operations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}