Training Configuration Options in OlmOCR: A Complete Guide to Model, Dataset, and Hyperparameter Settings

OlmOCR's training pipeline relies on a hierarchical configuration system defined in olmocr/train/config.py that splits settings across four dataclasses—ModelConfig, DatasetConfig, TrainingConfig, and Config—enabling precise control over vision-language model architecture, data processing pipelines, optimizer selection, and experiment tracking.

The allenai/olmocr repository provides a comprehensive training framework for fine-tuning vision-language models on OCR tasks. Understanding the available training configuration options in OlmOCR is essential for customizing everything from model quantization and LoRA fine-tuning to distributed training strategies. All settings are validated through Python dataclasses before the training loop begins in olmocr/train/train.py, ensuring type safety and parameter consistency across the pipeline.

Configuration Architecture Overview

The configuration system is organized into four hierarchical sections, each represented by a dedicated dataclass in olmocr/train/config.py:

• ModelConfig – Controls vision-language model selection, quantization, attention implementations, and LoRA parameters.
• DatasetConfig – Defines training and evaluation datasets, including root directories and preprocessing pipeline steps.
• TrainingConfig – Contains all hyperparameters for the training loop, including optimizer choice, learning rate scheduling, checkpointing, and logging.
• Config – The top-level wrapper that combines the three sections above with experiment metadata like Weights & Biases project names and distributed training settings.

Configurations are typically loaded from YAML files (such as configs/default_config.yaml) using Config.from_yaml() and validated via Config.validate() before training initiates.

Model Configuration (ModelConfig)

The ModelConfig dataclass governs which vision-language model is instantiated and how it is optimized for training. By default, OlmOCR uses Qwen/Qwen2.5-VL-7B-Instruct, though any compatible model can be specified via the name field.

Key configuration options include:

• Precision and Quantization – torch_dtype (e.g., bfloat16), load_in_8bit, and load_in_4bit for memory-efficient training.
• Attention Mechanisms – use_flash_attention and attn_implementation for optimized attention computation.
• Component Freezing – freeze_vision_tower and freeze_language_model to lock specific model components during training.
• LoRA Fine-Tuning – When use_lora is enabled, parameters include lora_rank, lora_alpha, lora_dropout, lora_target_modules, and lora_modules_to_save.

These settings are applied when the training script constructs the model in olmocr/train/train.py, with validation occurring in config.py before instantiation.

Dataset Configuration (DatasetConfig)

The DatasetConfig dataclass manages data sources through its train and eval fields, each accepting a list of dataset descriptor dictionaries. Every descriptor must specify a root_dir and a pipeline definition.

The pipeline is a critical concept in OlmOCR: it is a list of step dictionaries that define how raw PDFs are transformed into training examples. When BaseMarkdownPDFDataset is instantiated in olmocr/train/dataloader.py, the configuration calls Config.get_pipeline_steps() (lines 74-84 of config.py) to convert these dictionaries into concrete pipeline step objects.

Available pipeline steps include:

• FrontMatterParser – Extracts metadata from PDF front matter.
• PDFRenderer – Renders PDF pages to images.
• Tokenizer – Processes text into model-compatible token sequences.
• RandomTokenFlipper – Applies augmentation through token-level noise.

Each step has a corresponding *Config subclass that supplies default parameters, allowing granular control over preprocessing behavior without modifying source code.

Training Hyperparameters (TrainingConfig)

The TrainingConfig dataclass contains the most extensive set of training configuration options in OlmOCR, covering the entire optimization lifecycle.

Optimizer Settings

The optim field determines whether to use AdamW (default) or the custom Muon optimizer. When selecting Muon, additional parameters become available:

• muon_lr_multiplier_head – Learning rate multiplier for head parameters.
• muon_lr_multiplier_embed – Learning rate multiplier for embedding parameters.
• muon_lr_multiplier_scalar – Learning rate multiplier for scalar parameters.

For AdamW, standard parameters include adam_beta1, adam_beta2, adam_epsilon, weight_decay, and max_grad_norm.

Learning Rate and Scheduling

Configuration options include learning_rate, lr_scheduler_type (e.g., cosine, linear), warmup_ratio, and lr_scheduler_kwargs for scheduler-specific arguments.

Checkpointing and Evaluation

Control model persistence through save_strategy (e.g., steps, epoch), save_steps, save_total_limit, and load_best_model_at_end. Evaluation cadence is managed via evaluation_strategy and eval_steps.

Advanced Optimizations

• Gradient Checkpointing – Enabled via gradient_checkpointing with optional gradient_checkpointing_kwargs to trade compute for memory.
• Torch Compile – Setting torch_compile: true activates torch.compile() with configurable torch_compile_backend, torch_compile_mode, torch_compile_fullgraph, and torch_compile_dynamic flags (see lines 33-42 of train.py).
• Early Stopping – Configure use_early_stopping, early_stopping_patience, and early_stopping_threshold to halt training when metrics plateau on the criterion specified by metric_for_best_model.

Logging and Reproducibility

The report_to list determines which experiment trackers receive logs (e.g., wandb, tensorboard). When "wandb" is present, the script initializes Weights & Biases using wandb_project and wandb_entity from the top-level config. Reproducibility is managed through seed and data_seed, while dataloader_num_workers and dataloader_drop_last control data loading behavior.

Loading and Customizing Configurations

Configurations are loaded from YAML and can be inspected or modified programmatically before training.

Load and Inspect a Configuration

from olmocr.train.config import Config

# Load the default configuration

cfg = Config.from_yaml("configs/default_config.yaml")

print("Model:", cfg.model.name)
print("Training epochs:", cfg.training.num_train_epochs)
print("Batch size (train):", cfg.training.per_device_train_batch_size)
print("Optimizer:", cfg.training.optim)
print("LoRA enabled:", cfg.model.use_lora)

Override Settings Programmatically

from olmocr.train.config import Config, ModelConfig, TrainingConfig, DatasetConfig

# Start from the default config

cfg = Config.from_yaml("configs/default_config.yaml")

# Change model and training hyper-parameters

cfg.model.name = "Qwen/Qwen2.5-VL-7B-Instruct"
cfg.model.use_lora = True
cfg.training.num_train_epochs = 3
cfg.training.learning_rate = 5e-5
cfg.training.logging_steps = 20

# Add a new training dataset

cfg.dataset.train.append(
    {
        "root_dir": "/data/my_olmocr_dataset",
        "pipeline": [
            {"name": "FrontMatterParser"},
            {"name": "PDFRenderer"},
            {"name": "Tokenizer"},
        ],
    }
)

# Save the modified configuration for later runs

cfg.to_yaml("configs/custom_run.yaml")

Launch Training with a Custom Config

python -m olmocr.train.train --config configs/custom_run.yaml

Summary

• Four-dataclass hierarchy – ModelConfig, DatasetConfig, TrainingConfig, and Config in olmocr/train/config.py provide structured control over all training aspects.
• Default model – Qwen/Qwen2.5-VL-7B-Instruct with support for 8-bit/4-bit quantization and Flash Attention.
• Flexible pipelines – Dataset preprocessing is configurable through composable pipeline steps defined in YAML and instantiated via get_pipeline_steps().
• Optimizer choice – Support for both AdamW and the custom Muon optimizer with parameter-specific learning rate multipliers.
• Production features – Built-in support for gradient checkpointing, Torch compile acceleration, early stopping, and Weights & Biases logging.

Frequently Asked Questions

What is the default vision-language model in OlmOCR training configurations?

According to the ModelConfig dataclass in olmocr/train/config.py, the default model is Qwen/Qwen2.5-VL-7B-Instruct. This can be overridden by changing the name field in the configuration to any compatible Hugging Face model identifier.

How do I enable LoRA fine-tuning in OlmOCR?

Set use_lora: true in the model configuration section, then specify lora_rank, lora_alpha, lora_dropout, lora_target_modules, and lora_modules_to_save according to your fine-tuning requirements. These parameters control the rank decomposition and which model layers receive trainable adapter weights.

What optimizers are available besides AdamW?

OlmOCR supports the custom Muon optimizer in addition to standard AdamW. When optim is set to "muon", the training script uses Muon-specific learning rate multipliers (muon_lr_multiplier_head, muon_lr_multiplier_embed, muon_lr_multiplier_scalar) to handle different parameter groups, as implemented in the optimizer construction block of olmocr/train/train.py.

How do I configure early stopping to prevent overfitting?

Enable use_early_stopping: true in your training configuration, then set early_stopping_patience (number of evaluation steps to wait) and early_stopping_threshold (minimum improvement required). Ensure metric_for_best_model is set to the validation metric you want to monitor, and configure evaluation_strategy and eval_steps to establish how frequently the metric is computed.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}