# How to Fine-Tune olmOCR Models: A Complete Technical Guide > Learn to fine-tune olmOCR models with this technical guide. Prepare data, configure LoRA, train, and merge checkpoints for vLLM compatibility using the allenai olmocr repository. Get started now. - Repository: [Ai2/olmocr](https://github.com/allenai/olmocr) - Tags: how-to-guide - Published: 2026-07-07 --- **Fine-tuning olmOCR models requires preparing directories of single-page PDFs with matching Markdown files (containing YAML front-matter), configuring a YAML training file with LoRA settings, and executing the training pipeline followed by checkpoint merging for vLLM compatibility.** The olmOCR repository by AllenAI provides a production-grade framework for fine-tuning multimodal vision-language models on document understanding tasks. To successfully fine-tune olmOCR models, you must follow a specific data format and pipeline architecture that handles PDF rendering, document anchoring, and ChatML-style prompt generation. This guide explains the exact file structures, source code components, and commands used in the official training implementation. ## Data Preparation Requirements The training pipeline expects a strict pairing between PDF documents and Markdown annotation files. ### Directory Structure Create a directory where each single-page PDF has a corresponding `.md` file with the same base name: ```bash data/ ├── doc001.pdf # single-page PDF ├── doc001.md # markdown with YAML front-matter ├── doc002.pdf └── doc002.md ``` **Critical constraint:** Each PDF must contain exactly one page. The `validate_pdf_pair` function (lines 42-70 in [`olmocr/train/dataloader.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/dataloader.py)) enforces this validation during dataset initialization and will reject multi-page documents. ### Markdown Format Each `.md` file must begin with a YAML front-matter block containing document metadata followed by the extracted text: ```markdown --- primary_language: en is_rotation_valid: True rotation_correction: 0 is_table: False is_diagram: False --- Document text goes here extracted from the PDF... ``` ### Generating Training Datasets For large-scale dataset preparation, use the provided helper script to download and split the public olmOCR-mix dataset: ```bash python -m olmocr.data.prepare_olmocrmix \ --dataset-path allenai/olmOCR-mix-1025 \ --destination ~/olmOCR-mix-1025-extracted \ --subset 00_documents --split train ``` This script handles the conversion of multi-page documents into the required single-page format. ## Configuration Setup Training behavior is controlled through YAML configuration files parsed by `olmocr.train.config.Config.from_yaml` (implemented in [`olmocr/train/config.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/config.py)). Start with the official fine-tuning template: ```yaml project_name: olmocr-finetune run_name: custom-domain model: name: allenai/olmOCR-2-7B-1025 trust_remote_code: true torch_dtype: bfloat16 use_flash_attention: true attn_implementation: flash_attention_2 use_lora: true # Enable LoRA for efficient fine-tuning lora_rank: 8 lora_alpha: 32 lora_dropout: 0.1 lora_target_modules: [q_proj, v_proj, k_proj, o_proj] dataset: train: - name: custom_data root_dir: ./data # Path to your PDF/MD pairs pipeline: &basic - name: FrontMatterParser front_matter_class: PageResponse - name: PDFRenderer target_longest_image_dim: 1024 - name: StaticLengthDocumentAnchoring target_anchor_text_len: 4000 - name: NewYamlFinetuningPromptWithNoAnchoring - name: FrontMatterOutputFormat - name: InstructUserMessages prompt_first: true - name: Tokenizer masking_index: -100 end_of_message_token: "<|tool_call_end|>" training: output_dir: ./checkpoints num_train_epochs: 3 per_device_train_batch_size: 1 gradient_accumulation_steps: 4 learning_rate: 2e-5 collator_max_token_len: 8192 ``` **Key configuration fields:** - ** `model.use_lora`**: Set to `true` to train adapter weights instead of full model parameters - ** `dataset.train[*].root_dir`**: Absolute or relative path to your prepared data folder - ** `pipeline`**: Ordered list of transformation steps executed by `MarkdownPDFDocumentDataset` ## Running the Training Pipeline ### Installation Install the training dependencies with specific version constraints: ```bash pip install .[train] \ transformers==4.52.4 \ flash-attn>=2.8.0.post2 --no-build-isolation ``` ### Launching Training Execute the training entry point located at [`olmocr/train/train.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/train.py): ```bash python -m olmocr.train.train \ --config olmocr/train/configs/v0.4.0/qwen25_vl_olmocrv4_finetuning.yaml ``` The [`train.py`](https://github.com/allenai/olmocr/blob/main/train.py) script performs the following operations: 1. Loads the processor using `AutoProcessor.from_pretrained` 2. Instantiates `MarkdownPDFDocumentDataset` which walks the data folder, validates PDF/MD pairs, renders PDFs to images, and applies document anchoring 3. Builds ChatML-style messages through the pipeline steps defined in the config 4. Initializes the Hugging Face `Trainer` with the configured LoRA adapter (if enabled) 5. Writes checkpoints to `training.output_dir` ### Pipeline Architecture The `MarkdownPDFDocumentDataset` class (in [`olmocr/train/dataloader.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/dataloader.py)) orchestrates the data flow: - **FrontMatterParser**: Extracts metadata from YAML headers - **PDFRenderer**: Converts single-page PDFs to images at the specified resolution - **StaticLengthDocumentAnchoring**: Applies spatial anchoring to text regions - **Tokenizer**: Uses the Hugging Face `AutoProcessor` to generate `input_ids`, `pixel_values`, `attention_mask`, and masked `labels` (with `-100` for ignored tokens) ## Post-Processing and Deployment ### Merging LoRA Weights If you trained with LoRA (`use_lora: true`), merge the adapter weights into the base model for vLLM compatibility: ```bash python -m olmocr.train.prepare_checkpoint \ ./checkpoints/checkpoint-1000 \ ./merged_model ``` The [`prepare_checkpoint.py`](https://github.com/allenai/olmocr/blob/main/prepare_checkpoint.py) script loads the checkpoint, merges the LoRA matrices into the base model weights, and rewrites the model configuration to ensure compatibility with vLLM inference servers. ### FP8 Quantization (Optional) For optimized inference, convert the merged checkpoint to FP8 format: ```bash python -m olmocr.train.compress_checkpoint \ --config olmocr/train/quantization_configs/qwen2_5vl_w8a8_fp8.yaml \ ./merged_model \ ./fp8_model ``` This utilizes `torch.cuda.amp` calibration to reduce memory footprint while maintaining OCR accuracy. ### Serving with vLLM After merging, serve the model using vLLM: ```bash vllm serve ./merged_model --dtype bfloat16 ``` ## Summary - **Data format**: Single-page PDFs paired with Markdown files containing YAML front-matter, validated by `validate_pdf_pair` in [`olmocr/train/dataloader.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/dataloader.py) - **Configuration**: YAML files define the model, LoRA parameters, and pipeline steps, parsed by `olmocr.train.config.Config.from_yaml` - **Training**: Execute via `python -m olmocr.train.train` using the `MarkdownPDFDocumentDataset` pipeline to process documents through rendering, anchoring, and tokenization - **Post-processing**: Run [`prepare_checkpoint.py`](https://github.com/allenai/olmocr/blob/main/prepare_checkpoint.py) to merge LoRA weights and rewrite configs for vLLM compatibility - **Optimization**: Use [`compress_checkpoint.py`](https://github.com/allenai/olmocr/blob/main/compress_checkpoint.py) for optional FP8 quantization to reduce inference memory ## Frequently Asked Questions ### What is the exact data format required for fine-tuning olmOCR? You must provide pairs of files: a single-page PDF and a matching `.md` Markdown file with identical base names. The Markdown file must contain a YAML front-matter block (delimited by `---`) with fields like `primary_language`, `is_rotation_valid`, and `rotation_correction`, followed by the extracted document text. The validator in [`olmocr/train/dataloader.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/dataloader.py) enforces that each PDF contains exactly one page and rejects malformed pairs. ### How does the training pipeline process documents? The `MarkdownPDFDocumentDataset` class executes a sequential pipeline: it parses YAML front-matter, renders the PDF to an image using the PDFRenderer step, applies document anchoring via `StaticLengthDocumentAnchoring`, builds a ChatML-style prompt using functions from [`olmocr/prompts/prompts.py`](https://github.com/allenai/olmocr/blob/main/olmocr/prompts/prompts.py), and finally tokenizes the result using the Hugging Face AutoProcessor. This produces the `input_ids`, `pixel_values`, and masked `labels` required by the model. ### Can I use full fine-tuning instead of LoRA? Yes. Set `model.use_lora: false` in your configuration YAML. However, the repository defaults to LoRA (Low-Rank Adaptation) because it significantly reduces memory requirements and training time while maintaining performance. When using LoRA, you must run [`prepare_checkpoint.py`](https://github.com/allenai/olmocr/blob/main/prepare_checkpoint.py) after training to merge the adapter weights into the base model before serving with vLLM. ### How do I prepare checkpoints for inference after training? After training completes, use `python -m olmocr.train.prepare_checkpoint ` to merge LoRA weights (if applicable) and rewrite the model configuration. This script ensures the checkpoint is compatible with vLLM and standard Hugging Face pipelines. For production deployment, you can optionally run [`compress_checkpoint.py`](https://github.com/allenai/olmocr/blob/main/compress_checkpoint.py) to convert the model to FP8 format for faster inference and lower memory usage.