How to Set Up olmOCR Locally: Complete Installation Guide

Install Poppler utilities and Microsoft fonts, create a Python 3.11 environment, then run pip install "olmocr[gpu]" to deploy the local VLLM pipeline, or use pip install olmocr for remote-only inference.

Setting up olmOCR locally requires configuring system-level PDF rendering dependencies and Python packages that power the olmocr/pipeline.py orchestration layer. According to the allenai/olmocr source code, the tool modularizes heavy inference work to a separate VLLM server while the local pipeline handles PDF rendering, work-queue management, and result packaging. This guide covers Ubuntu/Debian prerequisites, GPU configurations, and both local and remote deployment options.

System Prerequisites

Ubuntu/Debian Packages

The rendering engine in olmocr/data/renderpdf.py relies on Poppler utilities and standard fonts to convert PDF pages into base64-encoded PNG images. Install these system dependencies before attempting Python package installation:

sudo apt-get update
sudo apt-get install poppler-utils ttf-mscorefonts-installer msttcorefonts \
    fonts-crosextra-caladea fonts-crosextra-carlito gsfonts lcdf-typetools

These packages enable the pipeline to handle font subsetting and page rasterization during the build_dolma_document workflow.

Hardware Requirements

While you can run olmOCR against a remote API without local acceleration, local GPU inference requires an NVIDIA GPU with at least 12 GB of VRAM (e.g., RTX 4090, A100). The VLLM server spawned by pipeline.py uses FlashAttention and optional FlashInfer optimizations to maximize throughput.

Python Environment Setup

Create an isolated Python 3.11 environment to avoid conflicts with PyTorch and CUDA dependencies:

conda create -n olmocr python=3.11
conda activate olmocr

Base Installation (Remote Only)

For setups using an external OpenAI-compatible endpoint (such as DeepInfra), install the lightweight base package:

pip install olmocr

Local GPU Installation

To run the allenai/olmOCR-2-7B-1025-FP8 model locally, install the GPU extras which bundle PyTorch 2.x with CUDA 12.8:

pip install "olmocr[gpu]" --extra-index-url https://download.pytorch.org/whl/cu128

For faster sampling speed, add the FlashInfer accelerator:

pip install https://download.pytorch.org/whl/cu128/flashinfer/flashinfer_python-0.2.5%2Bcu128torch2.7-cp38-abi3-linux_x86_64.whl

Optional Bundles

The pyproject.toml defines several extras for specific use cases:

• beaker: Submit jobs to the AI2 Beaker cluster using the --beaker flag
• bench: Install benchmark suite utilities via olmocr[bench]
• gpu,beaker: Combine local inference with Beaker cluster support

Install combined extras using: pip install "olmocr[gpu,beaker]".

Running Your First Conversion

Local GPU Pipeline

The pipeline automatically spawns a VLLM server when GPU dependencies are detected. Initialize your workspace and convert a PDF:

# Create workspace directory

mkdir -p ./localworkspace

# Download sample PDF

curl -o olmocr-sample.pdf https://olmocr.allenai.org/papers/olmocr_3pg_sample.pdf

# Convert using local GPU inference

olmocr ./localworkspace --markdown --pdfs olmocr-sample.pdf

This executes the entry point defined in olmocr/pipeline.py, which coordinates workers via olmocr/work_queue.py and writes Markdown output to ./localworkspace/markdown/ while storing Dolma-style JSONL records in ./localworkspace/results/.

Remote Server Configuration

To bypass local model loading and use an external VLLM endpoint, specify the server URL and API key:

olmocr ./localworkspace \
  --server http://my-vllm-host:8000/v1 \
  --model allenai/olmOCR-2-7B-1025-FP8 \
  --markdown \
  --pdfs *.pdf

The vllm_server_task function in pipeline.py detects the external endpoint and routes all page rendering requests there instead of spawning a local process.

Manual VLLM Server Launch

For debugging or multi-process setups, start the inference server manually before running the pipeline:

vllm serve allenai/olmOCR-2-7B-1025-FP8 \
  --port 8000 \
  --tensor-parallel-size 1 \
  --data-parallel-size 1 \
  --served-model-name olmocr \
  --disable-log-requests

The pipeline automatically detects http://localhost:8000/v1 and uses this endpoint for all PDF page processing.

Docker Alternative

Deploy a pre-built image containing all dependencies and the model weights (approximately 30 GB):

docker pull alleninstituteforai/olmocr:latest-with-model

docker run --gpus all \
  -v $(pwd):/workspace \
  alleninstituteforai/olmocr:latest-with-model \
  -c "olmocr /workspace/output --markdown --pdfs /workspace/sample.pdf"

This container includes the Poppler utilities, fonts, and Python environment configured in the repository's Dockerfile.with-model.

Summary

• System dependencies include poppler-utils, Microsoft TrueType fonts, and Crosextra fonts for PDF rendering in olmocr/data/renderpdf.py.
• Python 3.11 is required; use Conda to isolate the environment from system packages.
• Installation options: Base install (olmocr) for remote APIs, or GPU install (olmocr[gpu]) with CUDA 12.8 support for local inference.
• Execution: The olmocr CLI entry point in pipeline.py handles worker coordination, queue management via work_queue.py, and result packaging.
• Docker provides a fully containerized solution with the model weights pre-loaded.

Frequently Asked Questions

Do I need a GPU to run olmOCR locally?

No, but local GPU inference requires an NVIDIA GPU with at least 12 GB VRAM. You can run olmOCR on CPU-only machines by pointing the pipeline to a remote VLLM server using the --server flag, which routes all inference requests to an external endpoint without loading the model locally.

What Python version does olmOCR require?

The repository requires Python 3.11. Creating a dedicated Conda environment is strongly recommended because the GPU dependencies (PyTorch 2.x with CUDA 12.8) are large and may conflict with existing system packages.

How does the pipeline handle multiple PDFs concurrently?

The olmocr/pipeline.py module uses olmocr/work_queue.py to maintain a persistent queue of work items with file-based locking. This prevents duplicate processing when multiple workers access the same workspace, whether running locally or distributed across Beaker cluster nodes.

Where does olmOCR store conversion results?

By default, the pipeline writes Markdown files to {workspace}/markdown/ and Dolma-style JSONL records to {workspace}/results/. The build_dolma_document function in pipeline.py structures the output with extracted text and metadata, while get_markdown_path determines the final file paths for human-readable output.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}