# Performance Considerations for Open Code Review: Token Budgets and Resource Guards

> Discover performance considerations for Open Code Review including token budgets and resource guards. Optimize LLM API usage and prevent memory overflow with these essential features.

- Repository: [Alibaba/open-code-review](https://github.com/alibaba/open-code-review)
- Tags: performance
- Published: 2026-08-04

---

**Open Code Review implements deterministic token budgets, file size caps, and asynchronous compression to prevent LLM API quota exhaustion and memory overflow when scanning large repositories.**

Open Code Review (OCR) from Alibaba combines deterministic engineering with an LLM agent to deliver predictable performance in CI pipelines. Understanding the performance considerations for open-code-review is essential when scaling to large repositories, as the tool employs explicit safeguards against excessive resource consumption. These mechanisms ensure that token limits, memory usage, and API costs remain bounded regardless of codebase size.

## Token Budget Management and Hard Thresholds

OCR enforces a configurable **MAX_TOKENS** limit (default 58888) that governs every LLM request. The architecture implements a three-zone partitioning strategy defined in [`internal/agent/agent.go`](https://github.com/alibaba/open-code-review/blob/main/internal/agent/agent.go): normal operation below 60%, asynchronous compression at the **tokenSoftThreshold** (60%), and synchronous intervention at the **tokenWarningThreshold** (80%). When the token count exceeds 80% of the budget, the request is blocked entirely to prevent API quota waste.

Before any LLM invocation, OCR computes the token count of the diff. If the diff alone exceeds 80% of `MAX_TOKENS`, the file is skipped and reported as a warning. This fail-fast check prevents huge diffs—such as autogenerated lock files—from consuming API quota.

## File Size Caps and Binary Detection

Before any content reaches the LLM, [`internal/scan/provider.go`](https://github.com/alibaba/open-code-review/blob/main/internal/scan/provider.go) enforces a **DefaultMaxFileSizeBytes** limit of 2 MiB per file. Files exceeding this threshold are skipped with a warning, preventing multi-megabyte dumps from saturating the token budget.

Additionally, the **isBinaryFile** check identifies binary files and emits them as placeholders without loading contents into memory. This eliminates unnecessary I/O overhead and keeps the scanner's memory footprint predictable.

## Git-Aware Enumeration and Directory Pruning

OCR optimizes filesystem traversal through Git-aware detection. When operating within a Git work-tree, the **Provider.listFilesViaGit** method invokes `git ls-files` (including untracked files), which respects `.gitignore` rules and avoids walking ignored subtrees.

In non-Git mode, **Provider.listFilesViaWalk** utilizes **filepath.SkipDir** to prune entire subdirectories dynamically. This selective traversal significantly reduces scanning time on large monorepos compared to naive recursive walks.

## Asynchronous Compression Strategy

To maintain responsiveness while managing context windows, OCR implements **runCompression** as a background job triggered at the soft threshold (60%). The main agent loop continues issuing tool calls while the compression task summarizes older messages.

If the warning threshold (80%) is reached before async completion, OCR pauses execution and forces synchronous compression. This guarantees the next request fits within **MAX_TOKENS**, ensuring deterministic latency even under heavy load.

## Parallel Sub-Agents and Bundle Processing

Large changesets are partitioned into independent "bundles" (such as locale files or module groups), each processed by separate LLM contexts. This divide-and-conquer strategy keeps per-bundle token usage bounded and enables concurrent review across multi-core machines, preventing single massive diffs from monopolizing resources.

## Configurable Limits and CLI Overrides

Users can adjust performance parameters through the configuration system defined in [`internal/config/template/template.go`](https://github.com/alibaba/open-code-review/blob/main/internal/config/template/template.go). The **MaxTokens** and **MaxTokensBudget** fields allow raising limits for models with extended context windows, though defaults remain conservative for cost predictability.

```bash

# Run a review with the default token budget (≈ 59 k tokens)

ocr review

# Increase the token budget if your LLM supports it (e.g., to 150 k)

ocr config set MAX_TOKENS 150000

# Disable the per-file size cap (dangerous on huge repos)

ocr review --max-file-size 0

```

## Summary

- **Token budgets** with 60% soft and 80% hard thresholds prevent API quota exhaustion and guarantee request viability.
- **File size caps** (2 MiB default) and binary detection in [`internal/scan/provider.go`](https://github.com/alibaba/open-code-review/blob/main/internal/scan/provider.go) reduce memory pressure and I/O overhead.
- **Git-aware enumeration** via `git ls-files` and `filepath.SkipDir` optimization minimizes filesystem traversal time.
- **Asynchronous compression** maintains throughput while respecting token limits, falling back to synchronous mode when necessary.
- **Parallel sub-agents** partition large changesets into concurrent bundles for scalable multi-core processing.

## Frequently Asked Questions

### What is the default token limit in Open Code Review?

OCR defaults to 58888 tokens per LLM request, configurable via the `MAX_TOKENS` environment variable or `ocr config set`. This conservative default ensures predictable costs across different LLM providers.

### How does OCR handle files larger than 2 MiB?

Files exceeding `DefaultMaxFileSizeBytes` (2 MiB) are skipped during the enumeration phase in [`internal/scan/provider.go`](https://github.com/alibaba/open-code-review/blob/main/internal/scan/provider.go) and reported as warnings. This prevents memory overflow and protects the token budget from being consumed by single massive files.

### What happens when token usage reaches 80% of the budget?

When the **tokenWarningThreshold** is crossed, OCR pauses the main execution loop and runs compression synchronously. This ensures the next LLM request fits within the configured `MAX_TOKENS` limit, preventing API errors and quota waste.

### Does Open Code Review work efficiently without Git?

Yes. While Git mode uses `git ls-files` for efficient listing, non-Git mode implements optimized filesystem walking with `filepath.SkipDir` to prune subdirectories dynamically. This ensures reasonable performance even in generic directory structures without `.gitignore` optimization.