How to Fine-Tune olmOCR Models: A Complete Technical Guide

Fine-tuning olmOCR models requires preparing directories of single-page PDFs with matching Markdown files (containing YAML front-matter), configuring a YAML training file with LoRA settings, and executing the training pipeline followed by checkpoint merging for vLLM compatibility.

The olmOCR repository by AllenAI provides a production-grade framework for fine-tuning multimodal vision-language models on document understanding tasks. To successfully fine-tune olmOCR models, you must follow a specific data format and pipeline architecture that handles PDF rendering, document anchoring, and ChatML-style prompt generation. This guide explains the exact file structures, source code components, and commands used in the official training implementation.

Data Preparation Requirements

The training pipeline expects a strict pairing between PDF documents and Markdown annotation files.

Directory Structure

Create a directory where each single-page PDF has a corresponding .md file with the same base name:

data/
├── doc001.pdf          # single-page PDF

├── doc001.md           # markdown with YAML front-matter

├── doc002.pdf
└── doc002.md

Critical constraint: Each PDF must contain exactly one page. The validate_pdf_pair function (lines 42-70 in olmocr/train/dataloader.py) enforces this validation during dataset initialization and will reject multi-page documents.

Markdown Format

Each .md file must begin with a YAML front-matter block containing document metadata followed by the extracted text:

---
primary_language: en
is_rotation_valid: True
rotation_correction: 0
is_table: False
is_diagram: False
---
Document text goes here extracted from the PDF...

Generating Training Datasets

For large-scale dataset preparation, use the provided helper script to download and split the public olmOCR-mix dataset:

python -m olmocr.data.prepare_olmocrmix \
  --dataset-path allenai/olmOCR-mix-1025 \
  --destination ~/olmOCR-mix-1025-extracted \
  --subset 00_documents --split train

This script handles the conversion of multi-page documents into the required single-page format.

Configuration Setup

Training behavior is controlled through YAML configuration files parsed by olmocr.train.config.Config.from_yaml (implemented in olmocr/train/config.py).

Start with the official fine-tuning template:

project_name: olmocr-finetune
run_name: custom-domain
model:
  name: allenai/olmOCR-2-7B-1025
  trust_remote_code: true
  torch_dtype: bfloat16
  use_flash_attention: true
  attn_implementation: flash_attention_2
  use_lora: true              # Enable LoRA for efficient fine-tuning

  lora_rank: 8
  lora_alpha: 32
  lora_dropout: 0.1
  lora_target_modules: [q_proj, v_proj, k_proj, o_proj]

dataset:
  train:
    - name: custom_data
      root_dir: ./data         # Path to your PDF/MD pairs

      pipeline: &basic
        - name: FrontMatterParser
          front_matter_class: PageResponse
        - name: PDFRenderer
          target_longest_image_dim: 1024
        - name: StaticLengthDocumentAnchoring
          target_anchor_text_len: 4000
        - name: NewYamlFinetuningPromptWithNoAnchoring
        - name: FrontMatterOutputFormat
        - name: InstructUserMessages
          prompt_first: true
        - name: Tokenizer
          masking_index: -100
          end_of_message_token: "<|tool_call_end|>"

training:
  output_dir: ./checkpoints
  num_train_epochs: 3
  per_device_train_batch_size: 1
  gradient_accumulation_steps: 4
  learning_rate: 2e-5
  collator_max_token_len: 8192

Key configuration fields:

• ** model.use_lora**: Set to true to train adapter weights instead of full model parameters
• ** dataset.train[*].root_dir**: Absolute or relative path to your prepared data folder
• ** pipeline**: Ordered list of transformation steps executed by MarkdownPDFDocumentDataset

Running the Training Pipeline

Installation

Install the training dependencies with specific version constraints:

pip install .[train] \
    transformers==4.52.4 \
    flash-attn>=2.8.0.post2 --no-build-isolation

Launching Training

Execute the training entry point located at olmocr/train/train.py:

python -m olmocr.train.train \
    --config olmocr/train/configs/v0.4.0/qwen25_vl_olmocrv4_finetuning.yaml

The train.py script performs the following operations:

1. Loads the processor using AutoProcessor.from_pretrained
2. Instantiates MarkdownPDFDocumentDataset which walks the data folder, validates PDF/MD pairs, renders PDFs to images, and applies document anchoring
3. Builds ChatML-style messages through the pipeline steps defined in the config
4. Initializes the Hugging Face Trainer with the configured LoRA adapter (if enabled)
5. Writes checkpoints to training.output_dir

Pipeline Architecture

The MarkdownPDFDocumentDataset class (in olmocr/train/dataloader.py) orchestrates the data flow:

• FrontMatterParser: Extracts metadata from YAML headers
• PDFRenderer: Converts single-page PDFs to images at the specified resolution
• StaticLengthDocumentAnchoring: Applies spatial anchoring to text regions
• Tokenizer: Uses the Hugging Face AutoProcessor to generate input_ids, pixel_values, attention_mask, and masked labels (with -100 for ignored tokens)

Post-Processing and Deployment

Merging LoRA Weights

If you trained with LoRA (use_lora: true), merge the adapter weights into the base model for vLLM compatibility:

python -m olmocr.train.prepare_checkpoint \
    ./checkpoints/checkpoint-1000 \
    ./merged_model

The prepare_checkpoint.py script loads the checkpoint, merges the LoRA matrices into the base model weights, and rewrites the model configuration to ensure compatibility with vLLM inference servers.

FP8 Quantization (Optional)

For optimized inference, convert the merged checkpoint to FP8 format:

python -m olmocr.train.compress_checkpoint \
    --config olmocr/train/quantization_configs/qwen2_5vl_w8a8_fp8.yaml \
    ./merged_model \
    ./fp8_model

This utilizes torch.cuda.amp calibration to reduce memory footprint while maintaining OCR accuracy.

Serving with vLLM

After merging, serve the model using vLLM:

vllm serve ./merged_model --dtype bfloat16

Summary

• Data format: Single-page PDFs paired with Markdown files containing YAML front-matter, validated by validate_pdf_pair in olmocr/train/dataloader.py
• Configuration: YAML files define the model, LoRA parameters, and pipeline steps, parsed by olmocr.train.config.Config.from_yaml
• Training: Execute via python -m olmocr.train.train using the MarkdownPDFDocumentDataset pipeline to process documents through rendering, anchoring, and tokenization
• Post-processing: Run prepare_checkpoint.py to merge LoRA weights and rewrite configs for vLLM compatibility
• Optimization: Use compress_checkpoint.py for optional FP8 quantization to reduce inference memory

Frequently Asked Questions

What is the exact data format required for fine-tuning olmOCR?

You must provide pairs of files: a single-page PDF and a matching .md Markdown file with identical base names. The Markdown file must contain a YAML front-matter block (delimited by ---) with fields like primary_language, is_rotation_valid, and rotation_correction, followed by the extracted document text. The validator in olmocr/train/dataloader.py enforces that each PDF contains exactly one page and rejects malformed pairs.

How does the training pipeline process documents?

The MarkdownPDFDocumentDataset class executes a sequential pipeline: it parses YAML front-matter, renders the PDF to an image using the PDFRenderer step, applies document anchoring via StaticLengthDocumentAnchoring, builds a ChatML-style prompt using functions from olmocr/prompts/prompts.py, and finally tokenizes the result using the Hugging Face AutoProcessor. This produces the input_ids, pixel_values, and masked labels required by the model.

Can I use full fine-tuning instead of LoRA?

Yes. Set model.use_lora: false in your configuration YAML. However, the repository defaults to LoRA (Low-Rank Adaptation) because it significantly reduces memory requirements and training time while maintaining performance. When using LoRA, you must run prepare_checkpoint.py after training to merge the adapter weights into the base model before serving with vLLM.

How do I prepare checkpoints for inference after training?

After training completes, use python -m olmocr.train.prepare_checkpoint <checkpoint_dir> <output_dir> to merge LoRA weights (if applicable) and rewrite the model configuration. This script ensures the checkpoint is compatible with vLLM and standard Hugging Face pipelines. For production deployment, you can optionally run compress_checkpoint.py to convert the model to FP8 format for faster inference and lower memory usage.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}