# How to Set Up olmOCR Locally: Complete Installation Guide

> Install olmOCR locally with our comprehensive guide. Set up Python, install dependencies, and deploy the VLLM pipeline or use remote inference for efficient OCR.

- Repository: [Ai2/olmocr](https://github.com/allenai/olmocr)
- Tags: getting-started
- Published: 2026-07-07

---

**Install Poppler utilities and Microsoft fonts, create a Python 3.11 environment, then run `pip install "olmocr[gpu]"` to deploy the local VLLM pipeline, or use `pip install olmocr` for remote-only inference.**

Setting up olmOCR locally requires configuring system-level PDF rendering dependencies and Python packages that power the [`olmocr/pipeline.py`](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py) orchestration layer. According to the allenai/olmocr source code, the tool modularizes heavy inference work to a separate VLLM server while the local pipeline handles PDF rendering, work-queue management, and result packaging. This guide covers Ubuntu/Debian prerequisites, GPU configurations, and both local and remote deployment options.

## System Prerequisites

### Ubuntu/Debian Packages

The rendering engine in [`olmocr/data/renderpdf.py`](https://github.com/allenai/olmocr/blob/main/olmocr/data/renderpdf.py) relies on Poppler utilities and standard fonts to convert PDF pages into base64-encoded PNG images. Install these system dependencies before attempting Python package installation:

```bash
sudo apt-get update
sudo apt-get install poppler-utils ttf-mscorefonts-installer msttcorefonts \
    fonts-crosextra-caladea fonts-crosextra-carlito gsfonts lcdf-typetools

```

These packages enable the pipeline to handle font subsetting and page rasterization during the `build_dolma_document` workflow.

### Hardware Requirements

While you can run olmOCR against a remote API without local acceleration, **local GPU inference requires an NVIDIA GPU with at least 12 GB of VRAM** (e.g., RTX 4090, A100). The VLLM server spawned by [`pipeline.py`](https://github.com/allenai/olmocr/blob/main/pipeline.py) uses FlashAttention and optional FlashInfer optimizations to maximize throughput.

## Python Environment Setup

Create an isolated Python 3.11 environment to avoid conflicts with PyTorch and CUDA dependencies:

```bash
conda create -n olmocr python=3.11
conda activate olmocr

```

### Base Installation (Remote Only)

For setups using an external OpenAI-compatible endpoint (such as DeepInfra), install the lightweight base package:

```bash
pip install olmocr

```

### Local GPU Installation

To run the `allenai/olmOCR-2-7B-1025-FP8` model locally, install the GPU extras which bundle PyTorch 2.x with CUDA 12.8:

```bash
pip install "olmocr[gpu]" --extra-index-url https://download.pytorch.org/whl/cu128

```

For faster sampling speed, add the FlashInfer accelerator:

```bash
pip install https://download.pytorch.org/whl/cu128/flashinfer/flashinfer_python-0.2.5%2Bcu128torch2.7-cp38-abi3-linux_x86_64.whl

```

### Optional Bundles

The [`pyproject.toml`](https://github.com/allenai/olmocr/blob/main/pyproject.toml) defines several extras for specific use cases:

- **`beaker`**: Submit jobs to the AI2 Beaker cluster using the `--beaker` flag
- **`bench`**: Install benchmark suite utilities via `olmocr[bench]`
- **`gpu,beaker`**: Combine local inference with Beaker cluster support

Install combined extras using: `pip install "olmocr[gpu,beaker]"`.

## Running Your First Conversion

### Local GPU Pipeline

The pipeline automatically spawns a VLLM server when GPU dependencies are detected. Initialize your workspace and convert a PDF:

```bash

# Create workspace directory

mkdir -p ./localworkspace

# Download sample PDF

curl -o olmocr-sample.pdf https://olmocr.allenai.org/papers/olmocr_3pg_sample.pdf

# Convert using local GPU inference

olmocr ./localworkspace --markdown --pdfs olmocr-sample.pdf

```

This executes the entry point defined in [`olmocr/pipeline.py`](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py), which coordinates workers via [`olmocr/work_queue.py`](https://github.com/allenai/olmocr/blob/main/olmocr/work_queue.py) and writes Markdown output to `./localworkspace/markdown/` while storing Dolma-style JSONL records in `./localworkspace/results/`.

### Remote Server Configuration

To bypass local model loading and use an external VLLM endpoint, specify the server URL and API key:

```bash
olmocr ./localworkspace \
  --server http://my-vllm-host:8000/v1 \
  --model allenai/olmOCR-2-7B-1025-FP8 \
  --markdown \
  --pdfs *.pdf

```

The `vllm_server_task` function in [`pipeline.py`](https://github.com/allenai/olmocr/blob/main/pipeline.py) detects the external endpoint and routes all page rendering requests there instead of spawning a local process.

### Manual VLLM Server Launch

For debugging or multi-process setups, start the inference server manually before running the pipeline:

```bash
vllm serve allenai/olmOCR-2-7B-1025-FP8 \
  --port 8000 \
  --tensor-parallel-size 1 \
  --data-parallel-size 1 \
  --served-model-name olmocr \
  --disable-log-requests

```

The pipeline automatically detects `http://localhost:8000/v1` and uses this endpoint for all PDF page processing.

## Docker Alternative

Deploy a pre-built image containing all dependencies and the model weights (approximately 30 GB):

```bash
docker pull alleninstituteforai/olmocr:latest-with-model

docker run --gpus all \
  -v $(pwd):/workspace \
  alleninstituteforai/olmocr:latest-with-model \
  -c "olmocr /workspace/output --markdown --pdfs /workspace/sample.pdf"

```

This container includes the Poppler utilities, fonts, and Python environment configured in the repository's `Dockerfile.with-model`.

## Summary

- **System dependencies** include `poppler-utils`, Microsoft TrueType fonts, and Crosextra fonts for PDF rendering in [`olmocr/data/renderpdf.py`](https://github.com/allenai/olmocr/blob/main/olmocr/data/renderpdf.py).
- **Python 3.11** is required; use Conda to isolate the environment from system packages.
- **Installation options**: Base install (`olmocr`) for remote APIs, or GPU install (`olmocr[gpu]`) with CUDA 12.8 support for local inference.
- **Execution**: The `olmocr` CLI entry point in [`pipeline.py`](https://github.com/allenai/olmocr/blob/main/pipeline.py) handles worker coordination, queue management via [`work_queue.py`](https://github.com/allenai/olmocr/blob/main/work_queue.py), and result packaging.
- **Docker** provides a fully containerized solution with the model weights pre-loaded.

## Frequently Asked Questions

### Do I need a GPU to run olmOCR locally?

No, but local GPU inference requires an NVIDIA GPU with at least 12 GB VRAM. You can run olmOCR on CPU-only machines by pointing the pipeline to a remote VLLM server using the `--server` flag, which routes all inference requests to an external endpoint without loading the model locally.

### What Python version does olmOCR require?

The repository requires **Python 3.11**. Creating a dedicated Conda environment is strongly recommended because the GPU dependencies (PyTorch 2.x with CUDA 12.8) are large and may conflict with existing system packages.

### How does the pipeline handle multiple PDFs concurrently?

The [`olmocr/pipeline.py`](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py) module uses [`olmocr/work_queue.py`](https://github.com/allenai/olmocr/blob/main/olmocr/work_queue.py) to maintain a persistent queue of work items with file-based locking. This prevents duplicate processing when multiple workers access the same workspace, whether running locally or distributed across Beaker cluster nodes.

### Where does olmOCR store conversion results?

By default, the pipeline writes **Markdown files** to `{workspace}/markdown/` and **Dolma-style JSONL records** to `{workspace}/results/`. The `build_dolma_document` function in [`pipeline.py`](https://github.com/allenai/olmocr/blob/main/pipeline.py) structures the output with extracted text and metadata, while `get_markdown_path` determines the final file paths for human-readable output.