# AllenAI olmOCR Pre-trained Models: Available Checkpoints and Usage Guide

> Explore allenai/olmocr pre-trained models for OCR. Access FP8-quantized and FP16 checkpoints fine-tuned for efficient OCR tasks on Hugging Face. Get started today.

- Repository: [Ai2/olmocr](https://github.com/allenai/olmocr)
- Tags: usage-guide
- Published: 2026-07-07

---

**The allenai/olmocr repository provides multiple production-ready Vision-Language models on Hugging Face, including FP8-quantized fast variants and FP16 full-precision checkpoints based on Qwen2.5-VL, all fine-tuned specifically for OCR tasks.**

The allenai/olmocr project ships a suite of pre-trained models optimized for extracting text from documents. These checkpoints are fine-tuned versions of Qwen2.5-VL (and earlier Qwen2-VL) trained on synthetic OCR data, and they are hosted directly on the Hugging Face Hub for immediate download.

## Available Pre-trained Model Checkpoints

The repository maintains a clear release history documented in [`README.md`](https://github.com/allenai/olmocr/blob/main/README.md). Each version improves accuracy and inference speed on the olmOCR-Bench evaluation suite.

| Release | Model ID | Quantization | Key Improvements |
|---------|----------|--------------|------------------|
| **v0.4.0** (Oct 21, 2025) | `allenai/olmOCR-2-7B-1025-FP8` | FP8 | +4 points on olmOCR-Bench, synthetic data & RL training |
| **v0.3.0** (Aug 13, 2025) | `allenai/olmOCR-7B-0825-FP8` | FP8 | Fixed auto-rotation detection and blank-page hallucinations |
| **v0.2.1** (Jul 24, 2025) | `allenai/olmOCR-7B-0725-FP8` | FP8 | +3 points on olmOCR-Bench, faster inference |
| **v0.2.0** (Jul 23, 2025) | `allenai/olmOCR-7B-1025` | FP16 | First public finetuned checkpoint |
| **v0.1.75** (Jun 17, 2025) | `allenai/olmOCR-7B-1025` | FP16 | Switched to vLLM-based pipeline |

The **FP8** variants provide significantly faster inference with minimal accuracy loss, while **FP16** offers maximum precision for research use cases.

## Loading Models via the Hugging Face Transformers API

You can load any checkpoint directly using the `transformers` library. The repository includes a reference implementation in [`scripts/hf_local_test.py`](https://github.com/allenai/olmocr/blob/main/scripts/hf_local_test.py) that demonstrates proper initialization.

```python
import torch
from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration

model_id = "allenai/olmOCR-7B-0725-FP8"

processor = AutoProcessor.from_pretrained(model_id)
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
).eval()

# Process a PIL Image

inputs = processor(images=[pil_image], return_tensors="pt")
with torch.no_grad():
    output = model.generate(**inputs, max_new_tokens=1024)
text = processor.decode(output[0], skip_special_tokens=True)

```

This pattern uses `Qwen2_5_VLForConditionalGeneration` for model initialization and `AutoProcessor` for preprocessing, as implemented in the benchmark runner at [`olmocr/bench/runners/run_transformers.py`](https://github.com/allenai/olmocr/blob/main/olmocr/bench/runners/run_transformers.py).

## Local Inference with the olmOCR CLI

The `olmocr` command-line tool defaults to the most recent FP8 checkpoint but allows you to specify any model using the `--model` flag. The pipeline logic resides in [`olmocr/pipeline.py`](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py), which handles processor and model instantiation.

```bash
olmocr ./workspace \
  --model allenai/olmOCR-7B-0825-FP8 \
  --markdown \
  --pdfs mydoc.pdf

```

To process multiple documents or override the default server endpoint, adjust the workspace path and PDF glob pattern accordingly.

## Deploying with vLLM for Remote Inference

For production deployments, serve the model using vLLM and connect the CLI to your inference server. The benchmark suite includes reference code in [`olmocr/bench/runners/run_server.py`](https://github.com/allenai/olmocr/blob/main/olmocr/bench/runners/run_server.py) for this architecture.

Start the server:

```bash
vllm serve allenai/olmOCR-2-7B-1025-FP8 --max-model-len 16384

```

Connect the client:

```bash
olmocr ./workspace \
  --server http://localhost:8000/v1 \
  --model allenai/olmOCR-2-7B-1025-FP8 \
  --markdown \
  --pdfs *.pdf

```

This decouples the heavy model inference from the document processing pipeline, enabling horizontal scaling across multiple GPUs.

## Model Architecture and Training Details

All olmOCR models originate from the Qwen2.5-VL vision-language architecture. The training script at [`olmocr/train/train.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/train.py) implements the fine-tuning loop on synthetic OCR data, using `model.save_pretrained()` and `from_pretrained()` for checkpoint management. The v0.4.0 release incorporates reinforcement learning (RL) training to boost benchmark scores by 4 points.

## Summary

- The allenai/olmocr repository provides five public checkpoints on Hugging Face, ranging from v0.1.75 to v0.4.0.
- Models are available in **FP8** (fast, quantized) and **FP16** (full precision) variants.
- Load models using standard `transformers` APIs with `Qwen2_5_VLForConditionalGeneration`.
- Use the CLI with `--model` to switch between checkpoints, or deploy via vLLM for server-side inference.
- Source files [`scripts/hf_local_test.py`](https://github.com/allenai/olmocr/blob/main/scripts/hf_local_test.py) and [`olmocr/pipeline.py`](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py) contain the reference implementations for model loading.

## Frequently Asked Questions

### What base model architecture does olmOCR use?

olmOCR models are fine-tuned from **Qwen2.5-VL** (and earlier Qwen2-VL) checkpoints. This architecture combines vision encoders with large language model decoders to process document images and extract structured text.

### Should I use the FP8 or FP16 variant?

Choose **FP8** for production workloads where speed is critical, as it offers significantly faster inference with minimal accuracy degradation. Use **FP16** when you need maximum precision for research or when evaluating subtle OCR quality differences.

### How do I override the default model in the olmOCR CLI?

Pass the `--model` flag followed by the Hugging Face model ID. For example: `--model allenai/olmOCR-7B-0825-FP8`. The default is hardcoded in the pipeline initialization but can be overridden per command invocation.

### Can I fine-tune these pre-trained models on custom data?

Yes. The training script at [`olmocr/train/train.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/train.py) supports loading any pre-trained checkpoint using `from_pretrained()`, allowing you to resume training on domain-specific datasets. Save your new weights with `save_pretrained()` to create custom model variants.