OlmOCR Configuration Options: Complete Guide to Model, Training, and Pipeline Settings

OlmOCR uses a hierarchical dataclass configuration system defined in olmocr/train/config.py that aggregates model architecture, dataset loading, training hyperparameters, and modular preprocessing pipeline steps via YAML or Python APIs.

The allenai/olmocr repository implements a structured, dataclass-driven configuration framework that controls vision-language model initialization, distributed training, and document processing pipelines. Understanding these configuration options allows you to customize quantization, learning rate schedules, augmentation strategies, and experiment tracking without modifying source code.

Top-Level Configuration Structure

The central Config dataclass in olmocr/train/config.py serves as the root container that aggregates specialized sub-configurations and provides serialization methods.

The Config object contains the following primary fields:

• model: Instance of ModelConfig (lines 77-98) controlling which vision-language model to load and how to initialize it
• dataset: Instance of DatasetConfig (lines 68-74) specifying training and evaluation data sources
• training: Instance of TrainingConfig (lines 108-163) containing all hyperparameters for the training loop
• project_name: Human-readable experiment identifier used for Weights & Biases and TensorBoard logging
• run_name: Optional specific run identifier that auto-generates if omitted
• experiment_tracker: Backend selection (tensorboard, wandb, or mlflow) for experiment logging
• distributed and local_rank: Flags controlling multi-GPU and multi-node training behavior

The class provides YAML helpers including from_yaml() (lines 85-105), to_yaml(), validate(), and to_dict() for loading, saving, and sanity-checking configurations.

Model Configuration Options

The ModelConfig dataclass (lines 77-98 of olmocr/train/config.py) controls vision-language model selection, quantization, and memory optimization.

Model Selection and Loading:

• name: HuggingFace model identifier (default: "Qwen/Qwen2.5-VL-7B-Instruct")
• trust_remote_code: Boolean flag to allow execution of remote model code (default: False)
• device_map: Device allocation strategy (default: "auto")
• torch_dtype: PyTorch data type specification (float16, bfloat16, or "auto")

Quantization and Memory:

• load_in_8bit and load_in_4bit: Boolean flags for quantization (mutually exclusive)
• use_flash_attention: Enables FlashAttention when supported (default: True)
• attn_implementation: Specific attention backend selection (flash_attention_2, sdpa, or eager)

Parameter-Efficient Fine-Tuning (LoRA):

• use_lora: Boolean to enable low-rank adaptation
• lora_rank: Rank dimension (default: 8)
• lora_alpha: Scaling parameter
• lora_dropout: Dropout probability
• lora_target_modules: Modules to apply LoRA
• lora_modules_to_save: Additional modules to train fully

Freezing Options:

• freeze_vision_tower: Freeze vision encoder parameters
• freeze_language_model: Freeze language model parameters

Dataset Configuration Options

Dataset configuration uses DatasetConfig (lines 68-74) and DatasetItemConfig (lines 57-66) to define data sources and sampling strategies.

The dataset field contains:

• train: List of dictionaries, each specifying a training dataset with root_dir, pipeline steps, and optional max_samples
• eval: List of validation datasets following the same schema as train

Each dataset item requires a root_dir path and a pipeline list describing the preprocessing steps to apply.

Training Hyper-Parameters

The TrainingConfig dataclass (lines 108-163 of olmocr/train/config.py) exposes comprehensive HuggingFace Trainer-compatible options organized by functional category.

Batch and Scaling:

• per_device_train_batch_size and per_device_eval_batch_size: Batch sizes per GPU
• gradient_accumulation_steps: Steps to accumulate before backpropagation
• num_train_epochs: Total training epochs

Optimization:

• optim: Optimizer selection (adamw_torch or muon)
• learning_rate: Initial learning rate (default scheduler is cosine)
• lr_scheduler_type: Learning rate schedule algorithm
• warmup_ratio: Fraction of steps for warmup
• weight_decay: L2 regularization coefficient
• max_grad_norm: Gradient clipping threshold
• muon_lr_multiplier_*: Muon-specific learning rate multipliers when using Muon optimizer

Memory and Performance:

• gradient_checkpointing: Enable to trade compute for GPU memory
• torch_compile: Enable PyTorch 2.0 compilation with torch_compile_backend and torch_compile_mode options

Checkpointing and Evaluation:

• save_strategy and save_steps: Control checkpoint frequency
• save_total_limit: Maximum checkpoints to retain
• evaluation_strategy and eval_steps: Validation frequency
• load_best_model_at_end: Automatically load best checkpoint
• metric_for_best_model: Metric used to determine "best" model

Logging and Reproducibility:

• report_to: List of reporting backends (default: ["wandb"])
• logging_steps: Frequency of logging updates
• seed and data_seed: Random seeds for reproducibility

Early Stopping:

• use_early_stopping: Enable early termination
• early_stopping_patience: Epochs to wait without improvement
• early_stopping_threshold: Minimum change to qualify as improvement

Pipeline Step Configuration

Data processing pipelines are configured through dataclasses inheriting from PipelineStepConfig, instantiated via Config.get_pipeline_steps (lines 374-500 of olmocr/train/config.py). Each step includes name and enabled fields.

Rendering and Parsing:

• PDFRendererConfig: Controls PDF rasterization with target_longest_image_dim (integer)
• FrontMatterParserConfig: Document metadata extraction with use_page_response_class (boolean)
• StaticLengthDocumentAnchoringConfig: Text anchoring with target_anchor_text_len (integer)

Augmentation:

• RotationAugmentationConfig: Random rotation with probability (float)
• AugraphyBasicAugmentationsConfig: Physical document degradations with probability (float)

Tokenization and Filtering:

• TokenizerStepConfig: Token-level processing with masking_index and end_of_message_token
• RandomTokenFlipperConfig: Noise injection with token_flip_rate and masking_index
• FilterOutRotatedDocumentsConfig: Removes rotated documents from training
• DatasetTextRuleFilterConfig: Content-based filtering rules

Output Formatting:

• JSONOutputFormatConfig: Structured JSON output
• FrontMatterOutputFormatConfig: Markdown frontmatter output
• InstructUserMessagesConfig: Chat formatting with prompt_first (boolean)

Table and LaTeX Processing:

• TableTransformationConfig: Table annotations via transformation (e.g., "annotate_dims")
• LatexBracketNormalizerConfig: LaTeX bracket standardization
• ReformatLatexBoldItalicConfig: LaTeX formatting normalization

Loading and Validating Configurations

The training script olmocr/train/train.py consumes configurations through a standardized lifecycle that ensures type safety and path validation.

1. CLI Parsing: Line 295 defines the --config argument, defaulting to olmocr/train/configs/example_config.yaml
2. Deserialization: Config.from_yaml(args.config) (lines 85-105) constructs the hierarchical object graph from YAML
3. Validation: config.validate() verifies dataset paths exist, output directories are writable, and quantization settings are compatible
4. Programmatic Generation: create_default_config() (lines 506-508) generates template configurations for customization

Practical Configuration Examples

Loading Configuration from YAML

from olmocr.train.config import Config

# Load user-provided configuration

cfg = Config.from_yaml("olmocr/train/configs/example_config.yaml")

# Validate paths and create required directories

cfg.validate()

# Access specific fields

print(f"Model: {cfg.model.name}")
print(f"Batch size: {cfg.training.per_device_train_batch_size}")

Creating Configurations Programmatically

from olmocr.train.config import create_default_config

# Generate configuration with default values

cfg = create_default_config()

# Modify specific options

cfg.training.learning_rate = 2e-5
cfg.training.num_train_epochs = 5
cfg.model.use_lora = True
cfg.model.lora_rank = 16

# Persist to YAML for editing or reuse

cfg.to_yaml("custom_olmocr_config.yaml")

Integrating with Training Scripts

import argparse
from olmocr.train.config import Config

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "--config",
        type=str,
        default="olmocr/train/configs/example_config.yaml"
    )
    args = parser.parse_args()
    
    # Load and validate

    cfg = Config.from_yaml(args.config)
    cfg.validate()
    
    # Configuration ready for training loop

    return cfg

Sample YAML Configuration

model:
  name: "Qwen/Qwen2.5-VL-7B-Instruct"
  load_in_4bit: true
  use_flash_attention: true
  torch_dtype: "bfloat16"

dataset:
  train:
    - root_dir: "/data/olmocr/training"
      pipeline:
        - name: "PDFRenderer"
          target_longest_image_dim: 1024
        - name: "FrontMatterParser"
          enabled: true
        - name: "Tokenizer"
          masking_index: -100
      max_samples: 100000

training:
  output_dir: "./checkpoints"
  num_train_epochs: 3
  per_device_train_batch_size: 2
  gradient_accumulation_steps: 4
  learning_rate: 3e-5
  optim: "adamw_torch"
  lr_scheduler_type: "cosine"
  warmup_ratio: 0.1
  report_to: ["wandb"]
  logging_steps: 20
  gradient_checkpointing: true
  torch_compile: true

project_name: "olmocr-experiment-1"
experiment_tracker: "wandb"

Summary

• Root Configuration: The Config dataclass in olmocr/train/config.py aggregates model, dataset, training, and metadata settings with YAML serialization support
• Model Customization: Use ModelConfig to select base models, enable 4-bit or 8-bit quantization, configure FlashAttention backends, and set LoRA parameters for efficient fine-tuning
• Training Control: TrainingConfig provides HuggingFace Trainer integration with optimizer selection (AdamW or Muon), learning rate scheduling, gradient checkpointing, and torch.compile support
• Pipeline Flexibility: Modular pipeline steps configured via get_pipeline_steps handle PDF rendering, document anchoring, augmentation, tokenization, and output formatting
• Validation Workflow: Load configurations via Config.from_yaml(), verify with config.validate(), and override programmatically using create_default_config() when needed

Frequently Asked Questions

How do I enable 4-bit quantization in OlmOCR?

Set model.load_in_4bit: true in your YAML configuration file. This corresponds to the load_in_4bit field in the ModelConfig dataclass (lines 77-98 of olmocr/train/config.py). Ensure you do not enable both load_in_8bit and load_in_4bit simultaneously, as these quantization flags are mutually exclusive.

What is the difference between the Muon and AdamW optimizers in OlmOCR?

The TrainingConfig supports adamw_torch (default) and muon via the optim field. Muon optimization includes specific learning rate multipliers (muon_lr_multiplier_*) and is implemented as an alternative to AdamW for potentially improved convergence on certain vision-language tasks. Configure this in the TrainingConfig section (lines 108-163 of olmocr/train/config.py).

How do I add custom augmentation steps to the OlmOCR pipeline?

Create a new dataclass inheriting from PipelineStepConfig in olmocr/train/config.py, then add it to the get_pipeline_steps method (lines 374-500). Reference your new step in the YAML pipeline list under dataset.train[].pipeline with name and enabled fields, plus any custom parameters your step requires.

Where does OlmOCR store experiment tracking configuration?

Experiment tracking is controlled by the experiment_tracker field (tensorboard, wandb, or mlflow) and project_name in the top-level Config class. The training script olmocr/train/train.py automatically sets WANDB_PROJECT environment variables from these fields when report_to includes "wandb" in the TrainingConfig.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}