How to Optimize olmOCR for Speed: 7 Configuration Changes That Double Throughput

You can optimize olmOCR for speed by reducing image resolution to 1024-1400px, increasing vLLM concurrency to 500-800 requests, using 4-bit quantization, and switching to local storage backends to eliminate S3 latency.

The allenai/olmocr repository provides a high-accuracy OCR pipeline that converts PDF pages to images and processes them through a vision language model. To optimize olmOCR for speed, you must address three critical bottlenecks: the PDF rendering layer in olmocr/data/renderpdf.py, the LLM request handling in olmocr/pipeline.py (lines 81-124), and the work-queue orchestration in olmocr/work_queue.py (lines 27-84).

Optimize PDF Rendering Speed

The render_pdf_to_base64png function called from pipeline.build_page_query renders each PDF page to a PNG using Poppler’s pdftoppm. This step can consume 30-50% of total processing time on high-resolution documents.

Reduce Image Resolution

The target_longest_image_dim parameter defaults to 2048 pixels, which is often unnecessary for OCR accuracy. Reducing this value to 1024-1400 cuts render time by 30-50% without degrading text recognition quality.

# In your pipeline configuration or direct call

from olmocr.data.renderpdf import render_pdf_to_base64png

b64_image = render_pdf_to_base64png(
    "document.pdf", 
    page_num=1, 
    target_longest_image_dim=1200  # Down from default 2048

)

Cache Rendered Pages

Many PDFs share identical pages (e.g., cover sheets or headers). Store rendered PNGs in a local cache keyed by PDF-page hash to avoid re-rendering during retries.

from pathlib import Path
import hashlib
from olmocr.data.renderpdf import render_pdf_to_base64png

CACHE_DIR = Path("/tmp/olmocr_page_cache")
CACHE_DIR.mkdir(parents=True, exist_ok=True)

def cached_render(pdf_path, page, dim):
    key = hashlib.sha1(f"{pdf_path}:{page}:{dim}".encode()).hexdigest()
    cache_file = CACHE_DIR / f"{key}.png.b64"
    if cache_file.is_file():
        return cache_file.read_text()
    # Fall back to normal rendering

    b64 = render_pdf_to_base64png(pdf_path, page, target_longest_image_dim=dim)
    cache_file.write_text(b64)
    return b64

Replace the call in build_page_query with cached_render(...) to eliminate redundant CPU cycles.

Parallelize Rendering

The pdf_render_max_workers_limit semaphore in olmocr/pipeline.py (line 87) defaults to cpu_count-2. On machines with many cores, raise this limit by setting the environment variable before importing the pipeline:

export BEAKER_ASSIGNED_CPU_COUNT=32  # Match your actual CPU count

This allows the render_pdf_to_base64png calls to execute concurrently across all available cores.

Accelerate LLM Inference

The process_page → try_single_page → apost chain in olmocr/pipeline.py handles network round-trips to the vLLM server. This is typically the dominant latency source in GPU-rich environments.

Tune vLLM Concurrency

The max_concurrent_requests_limit (line 88) defaults to 1 and is later overwritten by --max_concurrent_requests. For GPUs like the A100 that can sustain many parallel prompts, increase this to 500-800:

python -m olmocr.pipeline \
  --max_concurrent_requests 600 \
  # ... other args

Use Quantized Models

The default model allenai/olmOCR-2-7B-1025-FP8 uses 8-bit quantization. Switching to the 4-bit version (allenai/olmOCR-2-7B-1025-FP4) halves memory bandwidth and increases throughput significantly:

--model allenai/olmOCR-2-7B-1025-FP4

Reduce Temperature Retries

The TEMPERATURE_BY_ATTEMPT schedule (lines 84-86) retries failed pages up to 8 times with increasing temperature. In practice, 3-4 attempts are sufficient. Lower args.max_page_retries from the default 8 to 4-5 to save compute and network time:

--max_page_retries 4

Batch Multiple Pages

vLLM currently processes one page per request. If your model supports longer contexts (16k+ tokens), modify build_page_query to batch 2-3 pages into a single prompt, reducing HTTP overhead by 50-66%.

Streamline Work Queue Orchestration

The WorkQueue class in olmocr/work_queue.py manages distributed processing and can throttle throughput through lock contention and S3 latency.

Increase Group Size

The --pages_per_group argument controls how many PDF pages enter a single WorkItem. Larger groups (200-300 pages) reduce the frequency of S3 index updates and lock operations:

--pages_per_group 250

Use Local Backend

When running on a single node, use a file:// workspace path to invoke the LocalBackend (lines 38-91) instead of S3. This eliminates network latency for index loading, lock creation, and done-flag writes.

Tune Lock Timeouts

The default worker-lock timeout is 1800 seconds. For fast workers, decrease this to 300 seconds via --worker_lock_timeout_secs to prevent stale locks from blocking new work:

--worker_lock_timeout_secs 300

Pre-Populate the Queue

Run python -m olmocr.pipeline --populate_queue ... once to create work_index_list.csv.zstd. Subsequent runs only fetch new PDFs, minimizing redundant download_zstd_csv and expand_s3_glob operations.

Hardware-Specific Configurations

GPU-rich environments (≥2 x A100):

• --tensor_parallel_size=2
• --max_concurrent_requests=800
• --max_page_retries=4
• --pages_per_group=250

CPU-only (many cores):

• Set BEAKER_ASSIGNED_CPU_COUNT to total cores
• Raise pdf_render_max_workers_limit to match CPU count
• Use LocalBackend to avoid S3 latency

Spot instances / preemptible:

• Reduce --max_page_error_rate from 0.004 to fail fast
• Disable --markdown output until final runs to reduce I/O

Complete Optimization Command

This configuration targets maximum throughput on a dual-A100 setup:

python -m olmocr.pipeline \
  file:///local/workspace \
  --model allenai/olmOCR-2-7B-1025-FP4 \
  --max_concurrent_requests 600 \
  --pages_per_group 250 \
  --max_page_retries 4 \
  --workers 40 \
  --target_longest_image_dim 1200 \
  --worker_lock_timeout_secs 300 \
  --markdown false \
  --beaker false

Summary

• Reduce image resolution to 1024-1400px in render_pdf_to_base64png to cut rendering time by 30-50%.
• Increase vLLM concurrency to 500-800 requests and use 4-bit quantization (-FP4) to maximize GPU utilization.
• Cache rendered pages locally to avoid redundant CPU work on retries.
• Increase group sizes to 200-300 pages and use LocalBackend to minimize S3 overhead and lock contention.
• Lower retry limits to 4-5 attempts and reduce lock timeouts to 300 seconds for faster failure recovery.

Frequently Asked Questions

What is the fastest image resolution for olmOCR?

Setting target_longest_image_dim to 1024-1400 provides the best speed-to-accuracy ratio according to the allenai/olmocr source code. The default 2048px often exceeds requirements for text recognition and adds unnecessary rendering overhead via Poppler’s pdftoppm.

How many concurrent requests can vLLM handle in olmOCR?

The --max_concurrent_requests flag controls the max_concurrent_requests_limit semaphore in olmocr/pipeline.py (line 88). For A100 GPUs, values of 500-800 are sustainable, while the default of 1 severely underutilizes GPU memory bandwidth.

Should I use S3 or local storage for maximum speed?

For single-node deployments, local storage (file:// paths) is significantly faster. The LocalBackend implementation in olmocr/work_queue.py (lines 38-91) eliminates network latency for index operations, lock files, and result writes that plague S3-backed workspaces.

Can I optimize olmOCR for speed without a GPU?

Yes. On CPU-only machines, set BEAKER_ASSIGNED_CPU_COUNT to your total core count to maximize pdf_render_max_workers_limit, use the LocalBackend to avoid S3 latency, and consider reducing image resolution to 1024px to minimize the CPU time spent in render_pdf_to_base64png.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}