Performance Considerations for Open Code Review: Token Budgets and Resource Guards

Open Code Review implements deterministic token budgets, file size caps, and asynchronous compression to prevent LLM API quota exhaustion and memory overflow when scanning large repositories.

Open Code Review (OCR) from Alibaba combines deterministic engineering with an LLM agent to deliver predictable performance in CI pipelines. Understanding the performance considerations for open-code-review is essential when scaling to large repositories, as the tool employs explicit safeguards against excessive resource consumption. These mechanisms ensure that token limits, memory usage, and API costs remain bounded regardless of codebase size.

Token Budget Management and Hard Thresholds

OCR enforces a configurable MAX_TOKENS limit (default 58888) that governs every LLM request. The architecture implements a three-zone partitioning strategy defined in internal/agent/agent.go: normal operation below 60%, asynchronous compression at the tokenSoftThreshold (60%), and synchronous intervention at the tokenWarningThreshold (80%). When the token count exceeds 80% of the budget, the request is blocked entirely to prevent API quota waste.

Before any LLM invocation, OCR computes the token count of the diff. If the diff alone exceeds 80% of MAX_TOKENS, the file is skipped and reported as a warning. This fail-fast check prevents huge diffs—such as autogenerated lock files—from consuming API quota.

File Size Caps and Binary Detection

Before any content reaches the LLM, internal/scan/provider.go enforces a DefaultMaxFileSizeBytes limit of 2 MiB per file. Files exceeding this threshold are skipped with a warning, preventing multi-megabyte dumps from saturating the token budget.

Additionally, the isBinaryFile check identifies binary files and emits them as placeholders without loading contents into memory. This eliminates unnecessary I/O overhead and keeps the scanner's memory footprint predictable.

Git-Aware Enumeration and Directory Pruning

OCR optimizes filesystem traversal through Git-aware detection. When operating within a Git work-tree, the Provider.listFilesViaGit method invokes git ls-files (including untracked files), which respects .gitignore rules and avoids walking ignored subtrees.

In non-Git mode, Provider.listFilesViaWalk utilizes filepath.SkipDir to prune entire subdirectories dynamically. This selective traversal significantly reduces scanning time on large monorepos compared to naive recursive walks.

Asynchronous Compression Strategy

To maintain responsiveness while managing context windows, OCR implements runCompression as a background job triggered at the soft threshold (60%). The main agent loop continues issuing tool calls while the compression task summarizes older messages.

If the warning threshold (80%) is reached before async completion, OCR pauses execution and forces synchronous compression. This guarantees the next request fits within MAX_TOKENS, ensuring deterministic latency even under heavy load.

Parallel Sub-Agents and Bundle Processing

Large changesets are partitioned into independent "bundles" (such as locale files or module groups), each processed by separate LLM contexts. This divide-and-conquer strategy keeps per-bundle token usage bounded and enables concurrent review across multi-core machines, preventing single massive diffs from monopolizing resources.

Configurable Limits and CLI Overrides

Users can adjust performance parameters through the configuration system defined in internal/config/template/template.go. The MaxTokens and MaxTokensBudget fields allow raising limits for models with extended context windows, though defaults remain conservative for cost predictability.

# Run a review with the default token budget (≈ 59 k tokens)

ocr review

# Increase the token budget if your LLM supports it (e.g., to 150 k)

ocr config set MAX_TOKENS 150000

# Disable the per-file size cap (dangerous on huge repos)

ocr review --max-file-size 0

Summary

• Token budgets with 60% soft and 80% hard thresholds prevent API quota exhaustion and guarantee request viability.
• File size caps (2 MiB default) and binary detection in internal/scan/provider.go reduce memory pressure and I/O overhead.
• Git-aware enumeration via git ls-files and filepath.SkipDir optimization minimizes filesystem traversal time.
• Asynchronous compression maintains throughput while respecting token limits, falling back to synchronous mode when necessary.
• Parallel sub-agents partition large changesets into concurrent bundles for scalable multi-core processing.

Frequently Asked Questions

What is the default token limit in Open Code Review?

OCR defaults to 58888 tokens per LLM request, configurable via the MAX_TOKENS environment variable or ocr config set. This conservative default ensures predictable costs across different LLM providers.

How does OCR handle files larger than 2 MiB?

Files exceeding DefaultMaxFileSizeBytes (2 MiB) are skipped during the enumeration phase in internal/scan/provider.go and reported as warnings. This prevents memory overflow and protects the token budget from being consumed by single massive files.

What happens when token usage reaches 80% of the budget?

When the tokenWarningThreshold is crossed, OCR pauses the main execution loop and runs compression synchronously. This ensures the next LLM request fits within the configured MAX_TOKENS limit, preventing API errors and quota waste.

Does Open Code Review work efficiently without Git?

Yes. While Git mode uses git ls-files for efficient listing, non-Git mode implements optimized filesystem walking with filepath.SkipDir to prune subdirectories dynamically. This ensures reasonable performance even in generic directory structures without .gitignore optimization.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}