AllenAI olmOCR Pre-trained Models: Available Checkpoints and Usage Guide

The allenai/olmocr repository provides multiple production-ready Vision-Language models on Hugging Face, including FP8-quantized fast variants and FP16 full-precision checkpoints based on Qwen2.5-VL, all fine-tuned specifically for OCR tasks.

The allenai/olmocr project ships a suite of pre-trained models optimized for extracting text from documents. These checkpoints are fine-tuned versions of Qwen2.5-VL (and earlier Qwen2-VL) trained on synthetic OCR data, and they are hosted directly on the Hugging Face Hub for immediate download.

Available Pre-trained Model Checkpoints

The repository maintains a clear release history documented in README.md. Each version improves accuracy and inference speed on the olmOCR-Bench evaluation suite.

Release	Model ID	Quantization	Key Improvements
v0.4.0 (Oct 21, 2025)	allenai/olmOCR-2-7B-1025-FP8	FP8	+4 points on olmOCR-Bench, synthetic data & RL training
v0.3.0 (Aug 13, 2025)	allenai/olmOCR-7B-0825-FP8	FP8	Fixed auto-rotation detection and blank-page hallucinations
v0.2.1 (Jul 24, 2025)	allenai/olmOCR-7B-0725-FP8	FP8	+3 points on olmOCR-Bench, faster inference
v0.2.0 (Jul 23, 2025)	allenai/olmOCR-7B-1025	FP16	First public finetuned checkpoint
v0.1.75 (Jun 17, 2025)	allenai/olmOCR-7B-1025	FP16	Switched to vLLM-based pipeline

The FP8 variants provide significantly faster inference with minimal accuracy loss, while FP16 offers maximum precision for research use cases.

Loading Models via the Hugging Face Transformers API

You can load any checkpoint directly using the transformers library. The repository includes a reference implementation in scripts/hf_local_test.py that demonstrates proper initialization.

import torch
from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration

model_id = "allenai/olmOCR-7B-0725-FP8"

processor = AutoProcessor.from_pretrained(model_id)
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
).eval()

# Process a PIL Image

inputs = processor(images=[pil_image], return_tensors="pt")
with torch.no_grad():
    output = model.generate(**inputs, max_new_tokens=1024)
text = processor.decode(output[0], skip_special_tokens=True)

This pattern uses Qwen2_5_VLForConditionalGeneration for model initialization and AutoProcessor for preprocessing, as implemented in the benchmark runner at olmocr/bench/runners/run_transformers.py.

Local Inference with the olmOCR CLI

The olmocr command-line tool defaults to the most recent FP8 checkpoint but allows you to specify any model using the --model flag. The pipeline logic resides in olmocr/pipeline.py, which handles processor and model instantiation.

olmocr ./workspace \
  --model allenai/olmOCR-7B-0825-FP8 \
  --markdown \
  --pdfs mydoc.pdf

To process multiple documents or override the default server endpoint, adjust the workspace path and PDF glob pattern accordingly.

Deploying with vLLM for Remote Inference

For production deployments, serve the model using vLLM and connect the CLI to your inference server. The benchmark suite includes reference code in olmocr/bench/runners/run_server.py for this architecture.

Start the server:

vllm serve allenai/olmOCR-2-7B-1025-FP8 --max-model-len 16384

Connect the client:

olmocr ./workspace \
  --server http://localhost:8000/v1 \
  --model allenai/olmOCR-2-7B-1025-FP8 \
  --markdown \
  --pdfs *.pdf

This decouples the heavy model inference from the document processing pipeline, enabling horizontal scaling across multiple GPUs.

Model Architecture and Training Details

All olmOCR models originate from the Qwen2.5-VL vision-language architecture. The training script at olmocr/train/train.py implements the fine-tuning loop on synthetic OCR data, using model.save_pretrained() and from_pretrained() for checkpoint management. The v0.4.0 release incorporates reinforcement learning (RL) training to boost benchmark scores by 4 points.

Summary

• The allenai/olmocr repository provides five public checkpoints on Hugging Face, ranging from v0.1.75 to v0.4.0.
• Models are available in FP8 (fast, quantized) and FP16 (full precision) variants.
• Load models using standard transformers APIs with Qwen2_5_VLForConditionalGeneration.
• Use the CLI with --model to switch between checkpoints, or deploy via vLLM for server-side inference.
• Source files scripts/hf_local_test.py and olmocr/pipeline.py contain the reference implementations for model loading.

Frequently Asked Questions

What base model architecture does olmOCR use?

olmOCR models are fine-tuned from Qwen2.5-VL (and earlier Qwen2-VL) checkpoints. This architecture combines vision encoders with large language model decoders to process document images and extract structured text.

Should I use the FP8 or FP16 variant?

Choose FP8 for production workloads where speed is critical, as it offers significantly faster inference with minimal accuracy degradation. Use FP16 when you need maximum precision for research or when evaluating subtle OCR quality differences.

How do I override the default model in the olmOCR CLI?

Pass the --model flag followed by the Hugging Face model ID. For example: --model allenai/olmOCR-7B-0825-FP8. The default is hardcoded in the pipeline initialization but can be overridden per command invocation.

Can I fine-tune these pre-trained models on custom data?

Yes. The training script at olmocr/train/train.py supports loading any pre-trained checkpoint using from_pretrained(), allowing you to resume training on domain-specific datasets. Save your new weights with save_pretrained() to create custom model variants.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}