# How the CLI Handles Concurrent Parsing with Memory Limits in Code-Graph-RAG

> Discover how the Code-Graph-RAG CLI manages concurrent parsing with memory limits using ThreadPoolExecutor, BoundedASTCache, and semaphores for efficient resource control.

- Repository: [Vitali Avagyan/code-graph-rag](https://github.com/vitali87/code-graph-rag)
- Tags: internals
- Published: 2026-08-13

---

**The CLI orchestrates parallel file parsing using a `ThreadPoolExecutor`, a `BoundedASTCache` that evicts entries when memory thresholds are exceeded, and a counting semaphore that blocks new tasks when the remaining memory budget cannot accommodate additional AST trees.**

The `vitali87/code-graph-rag` repository provides a command-line tool designed to build knowledge graphs from large codebases without exhausting system RAM. By combining Python’s `concurrent.futures` module with an intelligent caching layer in [`codebase_rag/ast_cache.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/ast_cache.py), the tool maximizes parsing throughput while strictly respecting the user-defined memory ceiling specified via the `--memory-limit` flag.

## ThreadPoolExecutor and Worker Scaling

Parallel processing originates in [`codebase_rag/cli.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/cli.py), where the entry point initializes a `ThreadPoolExecutor`. The executor’s `max_workers` parameter is dynamically calculated from two inputs: the number of available CPU cores and the `--memory-limit` value supplied by the user. This calculation ensures that worker count scales with hardware capability but retreats when the available RAM cannot support many simultaneous operations.

Before submitting jobs to the pool, the CLI invokes `load_parsers()` from [`codebase_rag/parser_loader.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/parser_loader.py) to instantiate Tree-sitter language parsers. Each worker thread receives a dedicated parser instance, eliminating redundant initialization overhead and preventing race conditions during concurrent execution.

## BoundedASTCache Memory Management

The system reserves the majority of heap space for parsed abstract syntax trees (ASTs). To prevent unbounded growth, [`codebase_rag/ast_cache.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/ast_cache.py) implements the `BoundedASTCache` class, which tracks the byte size of every stored tree.

When a new AST is inserted, the cache aggregates the total resident memory of all entries. If this sum exceeds the limit defined in [`codebase_rag/constants/cli.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/constants/cli.py), the cache immediately evicts the oldest entries using a least-recently-used (LRU) policy until the total falls below the threshold. This mechanism guarantees that the CLI never retains more parsed data than the configured budget allows, regardless of how many files are processed.

## Semaphore-Based Task Throttling

Beyond caching, the CLI employs a `threading.Semaphore` to gate the submission of new parse tasks. Before each file is processed, the system estimates the memory required for that specific parse based on file size and language complexity.

The submitting thread acquires the semaphore for that estimated amount, effectively reserving memory budget before the parse begins. When the parse completes and the AST is either cached or discarded, the semaphore releases the reserved units, allowing subsequent tasks to proceed. If the remaining memory budget is insufficient, the thread blocks until other workers free enough resources, creating built-in backpressure that prevents oversubscription.

## CLI Configuration and Propagation

User control over resource consumption is centralized in [`codebase_rag/constants/cli.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/constants/cli.py), which defines the `--memory-limit` argument. This value, specified in megabytes, propagates through the initialization chain to both the `BoundedASTCache` constructor and the semaphore initialization logic in the CLI entry point.

The limit also integrates with [`codebase_rag/services/graph_service.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/services/graph_service.py), where the `_apply_memory_limit` function enforces the same RAM budget during Neo4j query execution, ensuring downstream database operations remain constrained by the user’s resource specifications.

```python

# Example: Run the CLI with a 2 GB memory ceiling

python -m codebase_rag.cli /path/to/repo --memory-limit 2048

```

## Graceful Fallback to Sequential Processing

When available memory drops below a safety threshold—typically the estimated size of the largest remaining file—the CLI disables the thread pool and falls back to sequential parsing. This conservative strategy prevents out-of-memory crashes on edge cases involving extremely large single files or sudden memory pressure from external processes, ensuring that the process remains stable even under constrained environments.

```python

# Simplified logic from codebase_rag/cli.py

if estimated_memory > remaining_budget:
    # Fall back to single-threaded parsing

    for file_path in remaining_files:
        parse_single_file(file_path)

```

## Summary

- **`ThreadPoolExecutor`** in [`codebase_rag/cli.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/cli.py) manages worker threads with count bounded by CPU cores and the `--memory-limit` parameter.
- **`BoundedASTCache`** actively tracks AST memory usage in [`codebase_rag/ast_cache.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/ast_cache.py) and evicts old entries when the configured limit is approached.
- **Semaphore gating** reserves memory budget before each parse, blocking new tasks until sufficient RAM is available to prevent oversubscription.
- **`--memory-limit`** argument defined in [`codebase_rag/constants/cli.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/constants/cli.py) propagates through the parsing and graph service layers for uniform resource enforcement.
- **Sequential fallback** triggers automatically when memory constraints cannot support additional concurrent operations.

## Frequently Asked Questions

### How does the CLI prevent out-of-memory errors when parsing large repositories?

The CLI prevents memory exhaustion through a three-layer defense: a `BoundedASTCache` that evicts least-recently-used trees when usage exceeds the budget, a `threading.Semaphore` that blocks new parse jobs until sufficient memory is available, and an automatic fallback to single-threaded processing when the remaining budget cannot accommodate the largest pending file.

### What determines the number of concurrent workers in the thread pool?

The `ThreadPoolExecutor` initializes with `max_workers` derived from both the system’s CPU count and the `--memory-limit` parameter. The calculation ensures that worker count increases with available processors but decreases if the memory budget is too small to support many simultaneous parses, as implemented in [`codebase_rag/cli.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/cli.py).

### Where is the memory limit configured and how is it enforced?

The `--memory-limit` argument is defined in [`codebase_rag/constants/cli.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/constants/cli.py) and enforced by `BoundedASTCache` in [`codebase_rag/ast_cache.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/ast_cache.py) for AST storage, and by the semaphore logic in [`codebase_rag/cli.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/cli.py) for task scheduling. The same limit extends to Neo4j queries through `_apply_memory_limit` in [`codebase_rag/services/graph_service.py`](https://github.com/vitali87/code-graph-rag/blob/main/codebase_rag/services/graph_service.py).

### Can the cache evict entries while a worker is still processing a file?

No, the cache only stores completed AST objects. While a worker parses a file, the tree exists only in local thread memory. Once parsing completes and the tree is submitted to the cache, the `BoundedASTCache` may immediately evict it if the insertion pushes total usage over the limit, ensuring the memory constraint is respected at all times while never interrupting active work.