# How Code‑Graph‑RAG Handles Large Codebases: Bounded Caching, Batch Ingestion, and Incremental Parsing

> Learn how Code-Graph-RAG efficiently handles large codebases using bounded caching, batch ingestion, and incremental parsing for predictable memory and latency. Explore the vitali87/code-graph-rag repository.

- Repository: [Vitali Avagyan/code-graph-rag](https://github.com/vitali87/code-graph-rag)
- Tags: how-to-guide
- Published: 2026-08-14

---

**Code‑Graph‑RAG uses a bounded LRU AST cache, multi‑pass incremental parsing, batched graph ingestion, and streaming file access to keep memory and latency predictable even in monorepos with tens of thousands of files.**

Code‑Graph‑RAG is an open‑source RAG system designed to index, query, and edit very large, polyglot repositories. Its scalability rests on several coordinated components that work together to prevent out‑of‑memory errors and maintain fast query performance. Below is a detailed breakdown of how each mechanism works, grounded in the actual source code from `vitali87/code-graph-rag`.

## Memory‑Bounded AST Caching with `BoundedASTCache`

The core defense against memory exhaustion is **`BoundedASTCache`** in [`codebase_rag/ast_cache.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/ast_cache.py). This LRU cache enforces two hard limits:

- **`CACHE_MAX_ENTRIES`**: Maximum number of AST objects stored
- **`CACHE_MAX_MEMORY_MB`**: Maximum total memory consumed by the cache

When either limit is exceeded, the cache evicts the oldest ASTs. Evicted files are not lost—`_load_ast_from_disk()` lazily re‑parses them on demand. This guarantees that even a repository with tens of thousands of files never exhausts RAM.

```python
from codebase_rag.ast_cache import BoundedASTCache

cache = BoundedASTCache()

# ASTs are automatically managed; old entries evicted when limits hit

ast = cache.get_or_parse("src/large_module.py", parse_fn)

```

## Incremental Multi‑Pass Parsing

The indexer in [`codebase_rag/graph_updater.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/graph_updater.py) runs three distinct passes to build a complete code graph:

1. **Tree‑sitter parsing** – Extract syntax nodes and basic symbols
2. **Type‑inference & relationship resolution** – Resolve imports, inheritance, and call edges
3. **Graph‑ingestion** – Write nodes and relationships to Memgraph

All files parsed in the current run are recorded in **`_parsed_files`** regardless of cache eviction. This ensures that later passes can re‑parse evicted files without losing cross‑file edges. The result is a robust incremental build system that handles partial cache misses gracefully.

## Batched Ingestion into Memgraph and Vector Stores

Writing millions of symbols to a graph database or vector store one‑by‑one would cripple performance. Code‑Graph‑RAG solves this with configurable batching:

| Store | Mechanism | Configuration |
|-------|-----------|---------------|
| Memgraph | `MEMGRAPH_BATCH_SIZE` (default 1000 nodes/edges per batch) | [`codebase_rag/config.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/config.py) |
| Vector store (Qdrant/Milvus) | `store_embedding_batch()` bulk insertion | [`codebase_rag/vector_store.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/vector_store.py) |

The `settings.resolve_batch_size()` helper in [`codebase_rag/config.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/config.py) allows users to tune batch sizes for their hardware.

```python
from codegraph_rag.sdk.graph_loader import GraphLoader

loader = GraphLoader(repo_path="/path/to/large/repo")
loader.load()  # Runs three passes with BoundedASTCache

loader.ingest_batch(batch_size=2000)  # Override default MEMGRAPH_BATCH_SIZE

```

## Large‑File Streaming via MCP Tools

The MCP‑based file‑access tools in [`codebase_rag/tools/tool_descriptions.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/tools/tool_descriptions.py) accept **`offset`** and **`limit`** arguments. This allows callers to request only a slice of a huge source file, preventing the entire file from loading into memory when only a portion is needed (e.g., for a diff or partial context).

```python

# Streaming a large file via the MCP API

await mcp_registry.read_file(
    "src/very_large_file.py",
    offset=500_000,   # Start at byte 500k

    limit=10_000      # Read only 10k bytes

)

```

## Configurable Hardware Tuning

All performance‑critical parameters are exposed through [`codebase_rag/config.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/config.py):

- `MEMGRAPH_BATCH_SIZE`
- `CACHE_MAX_ENTRIES`
- `CACHE_MAX_MEMORY_MB`

Users can adjust these for their deployment target—whether a laptop with 16 GB RAM or a server with 256 GB.

## Hybrid Front‑Ends for C/C++ and C#

When a repository contains C/C++ or C# files, optional **libclang** or **Roslyn** front‑ends can pre‑process macros and semantic facts. These are only enabled when the required toolchain is present; otherwise the pure tree‑sitter pipeline runs. This prevents crashes on huge repositories that lack native tooling.

The relevant methods in [`codebase_rag/graph_updater.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/graph_updater.py) are:
- `_run_cpp_frontend()`
- `_run_csharp_frontend()`

## Parallel and Debounced File‑System Watching

In watch mode, [`realtime_updater.py`](https://github.com/vitali87/code-graph-rag/blob/main/realtime_updater.py) debounces file‑system events and processes them in configurable batches. This prevents a flood of updates from overwhelming the indexer when many files change at once—common after large refactors or branch switches.

```python

# Watch mode with batched updates

python realtime_updater.py --repo-path /path/to/repo --batch-size 50

```

## Benchmarked Performance

The [`optimize/profile_io.py`](https://github.com/vitali87/code-graph-rag/blob/main/optimize/profile_io.py) benchmark demonstrates Code‑Graph‑RAG ingesting a monorepo with **10,000+ nodes and 20,000+ relationships** without OOM. Query latency stays low thanks to indexed graph traversal and vector‑search caching.

## Quick Start for Large Codebases

```bash

# 1. Start the bundled Memgraph+Qdrant stack (no Docker compose needed)

cgr daemon up

# 2. Index a large repository; BoundedASTCache activates automatically

cgr start --repo-path /path/to/huge-repo --update-graph

# 3. Query with confidence in stable memory use

cgr query "Find all functions that import `numpy`"

```

## Summary

- **`BoundedASTCache`** in [`codebase_rag/ast_cache.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/ast_cache.py) caps memory use with LRU eviction and lazy re‑parsing
- **Three‑pass incremental parsing** in [`codebase_rag/graph_updater.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/graph_updater.py) tracks parsed files across cache misses
- **Batched ingestion** via `MEMGRAPH_BATCH_SIZE` and `store_embedding_batch()` minimizes database round‑trips
- **Streaming file access** via MCP `offset`/`limit` parameters avoids loading massive files whole
- **Configurable batch sizes** in [`codebase_rag/config.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/config.py) let users tune for their hardware
- **Optional native front‑ends** for C/C++ and C# gracefully degrade when toolchains are absent

- **Debounced watching** in [`realtime_updater.py`](https://github.com/vitali87/code-graph-rag/blob/main/realtime_updater.py) protects against update floods

## Frequently Asked Questions

### What happens when the AST cache is full?

The **`BoundedASTCache`** evicts the least‑recently‑used ASTs based on the `CACHE_MAX_ENTRIES` and `CACHE_MAX_MEMORY_MB` limits. Evicted files are re‑parsed on demand via `_load_ast_from_disk()` when later passes need them, so no data is lost—only temporarily unloaded from memory.

### Can I tune batch sizes for my hardware?

Yes. Adjust `MEMGRAPH_BATCH_SIZE` in [`codebase_rag/config.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/config.py) or override it programmatically with `loader.ingest_batch(batch_size=...)`. The `settings.resolve_batch_size()` helper provides sensible defaults while allowing full customization.

### How does Code‑Graph‑RAG avoid loading multi‑gigabyte source files?

The MCP tools expose **`offset`** and **`limit`** parameters in [`codebase_rag/tools/tool_descriptions.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/tools/tool_descriptions.py). Callers request specific byte ranges, so the system never materializes an entire massive file in memory unless explicitly asked.

### Does watching a large repository cause re‑indexing storms?

No. The [`realtime_updater.py`](https://github.com/vitali87/code-graph-rag/blob/main/realtime_updater.py) watcher debounces events and processes them in batches. The `batch_size` argument configures how many changes are coalesced before triggering an incremental update, preventing CPU and I/O spikes during bulk file operations.