# How to Optimize olmOCR for Speed: 7 Configuration Changes That Double Throughput

> Learn how to optimize olmOCR for speed with 7 configuration changes. Double throughput by reducing resolution, increasing concurrency, using 4-bit quantization, and switching to local storage.

- Repository: [Ai2/olmocr](https://github.com/allenai/olmocr)
- Tags: performance
- Published: 2026-07-08

---

**You can optimize olmOCR for speed by reducing image resolution to 1024-1400px, increasing vLLM concurrency to 500-800 requests, using 4-bit quantization, and switching to local storage backends to eliminate S3 latency.**

The `allenai/olmocr` repository provides a high-accuracy OCR pipeline that converts PDF pages to images and processes them through a vision language model. To optimize olmOCR for speed, you must address three critical bottlenecks: the **PDF rendering layer** in [`olmocr/data/renderpdf.py`](https://github.com/allenai/olmocr/blob/main/olmocr/data/renderpdf.py), the **LLM request handling** in [`olmocr/pipeline.py`](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py) (lines 81-124), and the **work-queue orchestration** in [`olmocr/work_queue.py`](https://github.com/allenai/olmocr/blob/main/olmocr/work_queue.py) (lines 27-84).

## Optimize PDF Rendering Speed

The `render_pdf_to_base64png` function called from `pipeline.build_page_query` renders each PDF page to a PNG using Poppler’s `pdftoppm`. This step can consume 30-50% of total processing time on high-resolution documents.

### Reduce Image Resolution

The `target_longest_image_dim` parameter defaults to 2048 pixels, which is often unnecessary for OCR accuracy. Reducing this value to **1024-1400** cuts render time by 30-50% without degrading text recognition quality.

```python

# In your pipeline configuration or direct call

from olmocr.data.renderpdf import render_pdf_to_base64png

b64_image = render_pdf_to_base64png(
    "document.pdf", 
    page_num=1, 
    target_longest_image_dim=1200  # Down from default 2048

)

```

### Cache Rendered Pages

Many PDFs share identical pages (e.g., cover sheets or headers). Store rendered PNGs in a local cache keyed by PDF-page hash to avoid re-rendering during retries.

```python
from pathlib import Path
import hashlib
from olmocr.data.renderpdf import render_pdf_to_base64png

CACHE_DIR = Path("/tmp/olmocr_page_cache")
CACHE_DIR.mkdir(parents=True, exist_ok=True)

def cached_render(pdf_path, page, dim):
    key = hashlib.sha1(f"{pdf_path}:{page}:{dim}".encode()).hexdigest()
    cache_file = CACHE_DIR / f"{key}.png.b64"
    if cache_file.is_file():
        return cache_file.read_text()
    # Fall back to normal rendering

    b64 = render_pdf_to_base64png(pdf_path, page, target_longest_image_dim=dim)
    cache_file.write_text(b64)
    return b64

```

Replace the call in `build_page_query` with `cached_render(...)` to eliminate redundant CPU cycles.

### Parallelize Rendering

The `pdf_render_max_workers_limit` semaphore in [`olmocr/pipeline.py`](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py) (line 87) defaults to `cpu_count-2`. On machines with many cores, raise this limit by setting the environment variable before importing the pipeline:

```bash
export BEAKER_ASSIGNED_CPU_COUNT=32  # Match your actual CPU count

```

This allows the `render_pdf_to_base64png` calls to execute concurrently across all available cores.

## Accelerate LLM Inference

The `process_page` → `try_single_page` → `apost` chain in [`olmocr/pipeline.py`](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py) handles network round-trips to the vLLM server. This is typically the dominant latency source in GPU-rich environments.

### Tune vLLM Concurrency

The `max_concurrent_requests_limit` (line 88) defaults to 1 and is later overwritten by `--max_concurrent_requests`. For GPUs like the A100 that can sustain many parallel prompts, increase this to **500-800**:

```bash
python -m olmocr.pipeline \
  --max_concurrent_requests 600 \
  # ... other args

```

### Use Quantized Models

The default model `allenai/olmOCR-2-7B-1025-FP8` uses 8-bit quantization. Switching to the **4-bit version** (`allenai/olmOCR-2-7B-1025-FP4`) halves memory bandwidth and increases throughput significantly:

```bash
--model allenai/olmOCR-2-7B-1025-FP4

```

### Reduce Temperature Retries

The `TEMPERATURE_BY_ATTEMPT` schedule (lines 84-86) retries failed pages up to 8 times with increasing temperature. In practice, 3-4 attempts are sufficient. Lower `args.max_page_retries` from the default 8 to **4-5** to save compute and network time:

```bash
--max_page_retries 4

```

### Batch Multiple Pages

vLLM currently processes one page per request. If your model supports longer contexts (16k+ tokens), modify `build_page_query` to batch 2-3 pages into a single prompt, reducing HTTP overhead by 50-66%.

## Streamline Work Queue Orchestration

The `WorkQueue` class in [`olmocr/work_queue.py`](https://github.com/allenai/olmocr/blob/main/olmocr/work_queue.py) manages distributed processing and can throttle throughput through lock contention and S3 latency.

### Increase Group Size

The `--pages_per_group` argument controls how many PDF pages enter a single `WorkItem`. Larger groups (200-300 pages) reduce the frequency of S3 index updates and lock operations:

```bash
--pages_per_group 250

```

### Use Local Backend

When running on a single node, use a `file://` workspace path to invoke the `LocalBackend` (lines 38-91) instead of S3. This eliminates network latency for index loading, lock creation, and done-flag writes.

### Tune Lock Timeouts

The default worker-lock timeout is 1800 seconds. For fast workers, decrease this to **300 seconds** via `--worker_lock_timeout_secs` to prevent stale locks from blocking new work:

```bash
--worker_lock_timeout_secs 300

```

### Pre-Populate the Queue

Run `python -m olmocr.pipeline --populate_queue ...` once to create `work_index_list.csv.zstd`. Subsequent runs only fetch new PDFs, minimizing redundant `download_zstd_csv` and `expand_s3_glob` operations.

## Hardware-Specific Configurations

**GPU-rich environments (≥2 x A100):**
- `--tensor_parallel_size=2`
- `--max_concurrent_requests=800`
- `--max_page_retries=4`
- `--pages_per_group=250`

**CPU-only (many cores):**
- Set `BEAKER_ASSIGNED_CPU_COUNT` to total cores
- Raise `pdf_render_max_workers_limit` to match CPU count
- Use `LocalBackend` to avoid S3 latency

**Spot instances / preemptible:**
- Reduce `--max_page_error_rate` from 0.004 to fail fast
- Disable `--markdown` output until final runs to reduce I/O

## Complete Optimization Command

This configuration targets maximum throughput on a dual-A100 setup:

```bash
python -m olmocr.pipeline \
  file:///local/workspace \
  --model allenai/olmOCR-2-7B-1025-FP4 \
  --max_concurrent_requests 600 \
  --pages_per_group 250 \
  --max_page_retries 4 \
  --workers 40 \
  --target_longest_image_dim 1200 \
  --worker_lock_timeout_secs 300 \
  --markdown false \
  --beaker false

```

## Summary

- **Reduce image resolution** to 1024-1400px in `render_pdf_to_base64png` to cut rendering time by 30-50%.
- **Increase vLLM concurrency** to 500-800 requests and use 4-bit quantization (`-FP4`) to maximize GPU utilization.
- **Cache rendered pages** locally to avoid redundant CPU work on retries.
- **Increase group sizes** to 200-300 pages and use `LocalBackend` to minimize S3 overhead and lock contention.
- **Lower retry limits** to 4-5 attempts and reduce lock timeouts to 300 seconds for faster failure recovery.

## Frequently Asked Questions

### What is the fastest image resolution for olmOCR?

Setting `target_longest_image_dim` to **1024-1400** provides the best speed-to-accuracy ratio according to the `allenai/olmocr` source code. The default 2048px often exceeds requirements for text recognition and adds unnecessary rendering overhead via Poppler’s `pdftoppm`.

### How many concurrent requests can vLLM handle in olmOCR?

The `--max_concurrent_requests` flag controls the `max_concurrent_requests_limit` semaphore in [`olmocr/pipeline.py`](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py) (line 88). For A100 GPUs, values of **500-800** are sustainable, while the default of 1 severely underutilizes GPU memory bandwidth.

### Should I use S3 or local storage for maximum speed?

For single-node deployments, **local storage** (`file://` paths) is significantly faster. The `LocalBackend` implementation in [`olmocr/work_queue.py`](https://github.com/allenai/olmocr/blob/main/olmocr/work_queue.py) (lines 38-91) eliminates network latency for index operations, lock files, and result writes that plague S3-backed workspaces.

### Can I optimize olmOCR for speed without a GPU?

Yes. On CPU-only machines, set `BEAKER_ASSIGNED_CPU_COUNT` to your total core count to maximize `pdf_render_max_workers_limit`, use the `LocalBackend` to avoid S3 latency, and consider reducing image resolution to 1024px to minimize the CPU time spent in `render_pdf_to_base64png`.