How to Run the olmOCR OCR Process: Complete Guide to PDF-to-Markdown Conversion

To run the olmOCR OCR process, install the package with GPU support, then execute olmocr <workspace> --pdfs <paths> to launch a local VLLM server that converts scanned PDFs into clean Markdown and Dolma-compatible JSONL using the allenai/olmOCR-2-7B-1025-FP8 vision-language model.

The allenai/olmocr open-source repository provides a production-ready pipeline for extracting text from image-based documents. This guide covers how to run the olmOCR OCR process on a single machine, connect to remote inference servers, and scale across distributed workers using cloud storage.

Prerequisites and Installation

Before running the pipeline, install system libraries for PDF rendering and font handling.

# Ubuntu/Debian dependencies

sudo apt-get update && sudo apt-get install -y \
    poppler-utils ttf-mscorefonts-installer \
    msttcorefonts fonts-crosextra-caladea \
    fonts-crosextra-carlito gsfonts lcdf-typetools

Create a Python environment and install olmOCR with GPU extras. The [gpu] flag installs PyTorch, FlashInfer, and CUDA dependencies required for local inference.

conda create -n olmocr python=3.11
conda activate olmocr
pip install "olmocr[gpu]" --extra-index-url https://download.pytorch.org/whl/cu128

# Optional: Install FlashInfer for faster inference

pip install https://download.pytorch.org/whl/cu128/flashinfer/flashinfer_python-0.2.5%2Bcu128torch2.7-cp38-abi3-linux_x86_64.whl

Basic Local Execution

The primary entry point is the olmocr CLI, which wraps python -m olmocr.pipeline. You must specify a workspace directory (for intermediate files and outputs) and one or more PDF paths (glob patterns supported).

olmocr ./my_workspace \
    --markdown \
    --pdfs ./document.pdf \
    --model allenai/olmOCR-2-7B-1025-FP8 \
    --workers 8

When executed, olmocr/pipeline.py performs the following initialization sequence:

1. Parses CLI arguments and validates the workspace path
2. Downloads the vision-language model via download_model() to a local cache
3. Starts a VLLM server subprocess (vllm_server_host) on an available GPU
4. Instantiates a WorkQueue to track PDF processing status
5. Spawns an asynchronous worker pool (default 20 workers) that consumes items from the queue

Core Pipeline Architecture

Understanding the internal components helps optimize throughput and debug failures.

WorkQueue and Job Distribution

The WorkQueue class in olmocr/work_queue.py manages parallel execution across local filesystems or S3. It splits input PDFs into work items (configurable via --pages_per_group) and tracks completion via atomic lock files.

• LocalBackend: Uses filesystem-based locking with .lock files
• S3Backend: Uses S3 object metadata for distributed locking across machines

The queue writes a compressed index to work_index_list.csv.zstd in the workspace, enabling workers to resume interrupted jobs.

Main Orchestration Logic

The main() function in olmocr/pipeline.py coordinates the end-to-end flow. For each PDF assigned to a worker, it calls process_pdf(), which delegates to process_single_pdf() for page-level processing.

Key functions in the orchestration chain:

• process_pdf(): Handles PDF download (if S3) and page grouping
• process_single_pdf(): Iterates through pages and calls process_page() asynchronously
• build_dolma_document(): Stitches page results into a JSONL document with metadata fields including language detection, rotation angles, and table flags
• get_markdown_path(): Generates .md output paths mirroring the source PDF hierarchy when --markdown is enabled

Page Processing and VLLM Integration

Each page flows through process_page() in olmocr/pipeline.py:

1. Render: render_pdf_to_base64png() (from olmocr/data/renderpdf.py) converts the PDF page to a base64-encoded PNG
2. Prompt: build_page_query() constructs a VLLM request using build_no_anchoring_v4_yaml_prompt with the image as a vision input
3. Inference: apost() sends an HTTP POST to the VLLM server (or remote endpoint) using a lightweight async socket client (avoiding aiohttp dependencies)
4. Parse: The response YAML front-matter is parsed into a PageResponse object containing extracted text, confidence scores, and rotation metadata
5. Retry: On transient errors or rotation mismatches, the system retries with exponential backoff (configurable via --max_page_retries, default 8)

If the model fails repeatedly, make_fallback_result() triggers a pure pdftotext extraction via get_anchor_text() to ensure deterministic output.

Using Remote Inference Servers

To run olmOCR without local GPUs, use the --server flag to point to any OpenAI-compatible endpoint (vLLM, DeepInfra, or Cirrascale).

olmocr ./my_workspace \
    --markdown \
    --pdfs ./document.pdf \
    --server http://my-vllm:8000/v1 \
    --model allenai/olmOCR-2-7B-1025-FP8 \
    --workers 16

When --server is provided, the pipeline skips vllm_server_host initialization and delegates all inference to the remote URL. This mode requires no CUDA libraries on the local machine.

Scaling with S3 and Distributed Workers

For large-scale processing, specify an S3 URI as the workspace and use the --beaker flag to submit jobs to AI2's Beaker cluster.

olmocr s3://my-bucket/workspaces/ocr_run \
    --pdfs s3://my-bucket/pdfs/*.pdf \
    --workers 20 \
    --beaker \
    --beaker_gpus 4 \
    --beaker_cluster ai2/allennlp

Execution flow with S3:

• The first worker populates the queue by writing work_index_list.csv.zstd to the S3 prefix
• Workers acquire locks by writing flag objects to S3
• Each worker downloads PDFs to local temp space, processes them, and uploads results to s3://bucket/workspaces/ocr_run/results/
• Final Markdown files appear under s3://bucket/workspaces/ocr_run/markdown/

The submit_beaker_job function (see pipeline.py line 291) creates a Docker-based experiment using the official jakep/olmocr-inference image.

Output Formats and Verification

The pipeline produces two primary outputs in the workspace directory:

• Dolma JSONL: Located in results/, each line contains a document with id, text, metadata (rotation, language), and attributes (page spans). Generated by build_dolma_document().
• Markdown: When --markdown is set, .md files are written to markdown/ with the same relative path structure as the source PDFs, determined by get_markdown_path().

Verify output with:

cat my_workspace/markdown/document.md

# or

zcat my_workspace/results/documents.jsonl.zstd | head -1 | jq .

Summary

• Installation: Install system fonts, poppler-utils, and olmocr[gpu] via pip
• Local execution: Run olmocr <workspace> --pdfs <paths> to start a VLLM server and process PDFs with 20 parallel workers by default
• Architecture: The WorkQueue in olmocr/work_queue.py manages job distribution, while olmocr/pipeline.py handles VLLM orchestration and page-level processing
• Fallbacks: If the vision model fails, make_fallback_result() in pipeline.py extracts text using pdftotext
• Remote mode: Use --server to connect to external vLLM instances without local GPU requirements
• Distributed scaling: Use S3 workspaces with --beaker for cloud-based processing across multiple nodes

Frequently Asked Questions

What hardware is required to run olmOCR locally?

Running olmOCR locally requires a CUDA-capable GPU with sufficient VRAM to load the allenai/olmOCR-2-7B-1025-FP8 model. The pipeline automatically starts a VLLM server using vllm_server_host in olmocr/pipeline.py, which manages GPU memory and batching. For CPU-only environments, use the --server flag to connect to a remote inference endpoint.

How does olmOCR handle multi-page PDFs?

The process_single_pdf() function in olmocr/pipeline.py splits PDFs into individual pages, rendering each to PNG via render_pdf_to_base64png(). Pages are processed asynchronously by the worker pool, and build_dolma_document() reassembles them into a single output document. The --pages_per_group flag controls how many pages constitute a single work item in the queue.

Can I process PDFs without installing GPU drivers?

Yes. By specifying --server http://remote-url/v1, you can run the olmOCR OCR process on a CPU-only machine. The pipeline delegates all inference to the remote endpoint and skips the local VLLM server initialization. Ensure you set --workers appropriately to match the remote server's capacity.

What happens when the model fails to recognize a page?

The process_page() function implements retry logic with exponential backoff for transient network errors. If the model returns invalid YAML or exceeds --max_page_retries (default 8), the pipeline calls make_fallback_result() to extract text using pdftotext via get_anchor_text(). This ensures every page produces output, even if the vision model encounters unreadable scans.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}