How the CLI Handles Concurrent Parsing with Memory Limits in Code-Graph-RAG

The CLI orchestrates parallel file parsing using a ThreadPoolExecutor, a BoundedASTCache that evicts entries when memory thresholds are exceeded, and a counting semaphore that blocks new tasks when the remaining memory budget cannot accommodate additional AST trees.

The vitali87/code-graph-rag repository provides a command-line tool designed to build knowledge graphs from large codebases without exhausting system RAM. By combining Python’s concurrent.futures module with an intelligent caching layer in codebase_rag/ast_cache.py, the tool maximizes parsing throughput while strictly respecting the user-defined memory ceiling specified via the --memory-limit flag.

ThreadPoolExecutor and Worker Scaling

Parallel processing originates in codebase_rag/cli.py, where the entry point initializes a ThreadPoolExecutor. The executor’s max_workers parameter is dynamically calculated from two inputs: the number of available CPU cores and the --memory-limit value supplied by the user. This calculation ensures that worker count scales with hardware capability but retreats when the available RAM cannot support many simultaneous operations.

Before submitting jobs to the pool, the CLI invokes load_parsers() from codebase_rag/parser_loader.py to instantiate Tree-sitter language parsers. Each worker thread receives a dedicated parser instance, eliminating redundant initialization overhead and preventing race conditions during concurrent execution.

BoundedASTCache Memory Management

The system reserves the majority of heap space for parsed abstract syntax trees (ASTs). To prevent unbounded growth, codebase_rag/ast_cache.py implements the BoundedASTCache class, which tracks the byte size of every stored tree.

When a new AST is inserted, the cache aggregates the total resident memory of all entries. If this sum exceeds the limit defined in codebase_rag/constants/cli.py, the cache immediately evicts the oldest entries using a least-recently-used (LRU) policy until the total falls below the threshold. This mechanism guarantees that the CLI never retains more parsed data than the configured budget allows, regardless of how many files are processed.

Semaphore-Based Task Throttling

Beyond caching, the CLI employs a threading.Semaphore to gate the submission of new parse tasks. Before each file is processed, the system estimates the memory required for that specific parse based on file size and language complexity.

The submitting thread acquires the semaphore for that estimated amount, effectively reserving memory budget before the parse begins. When the parse completes and the AST is either cached or discarded, the semaphore releases the reserved units, allowing subsequent tasks to proceed. If the remaining memory budget is insufficient, the thread blocks until other workers free enough resources, creating built-in backpressure that prevents oversubscription.

CLI Configuration and Propagation

User control over resource consumption is centralized in codebase_rag/constants/cli.py, which defines the --memory-limit argument. This value, specified in megabytes, propagates through the initialization chain to both the BoundedASTCache constructor and the semaphore initialization logic in the CLI entry point.

The limit also integrates with codebase_rag/services/graph_service.py, where the _apply_memory_limit function enforces the same RAM budget during Neo4j query execution, ensuring downstream database operations remain constrained by the user’s resource specifications.

# Example: Run the CLI with a 2 GB memory ceiling

python -m codebase_rag.cli /path/to/repo --memory-limit 2048

Graceful Fallback to Sequential Processing

When available memory drops below a safety threshold—typically the estimated size of the largest remaining file—the CLI disables the thread pool and falls back to sequential parsing. This conservative strategy prevents out-of-memory crashes on edge cases involving extremely large single files or sudden memory pressure from external processes, ensuring that the process remains stable even under constrained environments.

# Simplified logic from codebase_rag/cli.py

if estimated_memory > remaining_budget:
    # Fall back to single-threaded parsing

    for file_path in remaining_files:
        parse_single_file(file_path)

Summary

• ThreadPoolExecutor in codebase_rag/cli.py manages worker threads with count bounded by CPU cores and the --memory-limit parameter.
• BoundedASTCache actively tracks AST memory usage in codebase_rag/ast_cache.py and evicts old entries when the configured limit is approached.
• Semaphore gating reserves memory budget before each parse, blocking new tasks until sufficient RAM is available to prevent oversubscription.
• --memory-limit argument defined in codebase_rag/constants/cli.py propagates through the parsing and graph service layers for uniform resource enforcement.
• Sequential fallback triggers automatically when memory constraints cannot support additional concurrent operations.

Frequently Asked Questions

How does the CLI prevent out-of-memory errors when parsing large repositories?

The CLI prevents memory exhaustion through a three-layer defense: a BoundedASTCache that evicts least-recently-used trees when usage exceeds the budget, a threading.Semaphore that blocks new parse jobs until sufficient memory is available, and an automatic fallback to single-threaded processing when the remaining budget cannot accommodate the largest pending file.

What determines the number of concurrent workers in the thread pool?

The ThreadPoolExecutor initializes with max_workers derived from both the system’s CPU count and the --memory-limit parameter. The calculation ensures that worker count increases with available processors but decreases if the memory budget is too small to support many simultaneous parses, as implemented in codebase_rag/cli.py.

Where is the memory limit configured and how is it enforced?

The --memory-limit argument is defined in codebase_rag/constants/cli.py and enforced by BoundedASTCache in codebase_rag/ast_cache.py for AST storage, and by the semaphore logic in codebase_rag/cli.py for task scheduling. The same limit extends to Neo4j queries through _apply_memory_limit in codebase_rag/services/graph_service.py.

Can the cache evict entries while a worker is still processing a file?

No, the cache only stores completed AST objects. While a worker parses a file, the tree exists only in local thread memory. Once parsing completes and the tree is submitted to the cache, the BoundedASTCache may immediately evict it if the insertion pushes total usage over the limit, ensuring the memory constraint is respected at all times while never interrupting active work.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}