# What Models Are Supported for SFT Training in olmOCR?

> Explore olmOCR's support for SFT training with Qwen3-VL, Qwen2.5-VL, and Qwen2-VL models. Discover automatic HuggingFace class selection for seamless integration.

- Repository: [Ai2/olmocr](https://github.com/allenai/olmocr)
- Tags: api-reference
- Published: 2026-07-02

---

**OlmOCR supports three Qwen vision-language model families for SFT training—Qwen3-VL, Qwen2.5-VL, and Qwen2-VL—automatically selecting the appropriate HuggingFace `*ForConditionalGeneration` class based on the model identifier string in your configuration.**

The `allenai/olmocr` repository provides a complete pipeline for supervised fine-tuning (SFT) of vision-language models on OCR tasks. The training system inspects the `name` field defined in the `ModelConfig` dataclass to instantiate the correct architecture, allowing seamless switching between compatible Qwen VL models without modifying the core training logic.

## Supported Vision-Language Model Families

The SFT training pipeline recognizes specific identifier patterns that map to distinct HuggingFace model classes. Each family offers different capabilities for document understanding and text extraction tasks.

### Qwen3-VL

Models containing `qwen3-vl` in their identifier instantiate `Qwen3VLForConditionalGeneration`. This represents the latest generation of Qwen's vision-language architecture with enhanced capabilities.

```yaml
model:
  name: "Qwen/Qwen3-VL-7B-Instruct"

```

### Qwen2.5-VL (Including OlmOCR Checkpoints)

The `qwen2.5-vl` substring or the specific `olmocr-2-7b-1025` identifier triggers `Qwen2_5_VLForConditionalGeneration`. This is the **default model family** used by the repository, with `Qwen/Qwen2.5-VL-7B-Instruct` serving as the standard checkpoint defined in [`olmocr/train/config.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/config.py) at lines 177-184.

This family includes the official OlmOCR pretrained weights when specified by the `olmocr-2-7b-1025` identifier.

### Qwen2-VL

Models matching `qwen2-vl` utilize `Qwen2VLForConditionalGeneration`. This architecture remains fully supported for backwards compatibility and specific deployment scenarios requiring this earlier generation.

```yaml
model:
  name: "Qwen/Qwen2-VL-7B-Instruct"

```

## How Model Selection Works

The dispatch logic resides in **[`olmocr/train/train.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/train.py)** at lines 330-343. The training script performs lowercase string matching on the model identifier to determine the correct class:

```python
if "qwen3-vl" in config.model.name.lower():
    model_class = Qwen3VLForConditionalGeneration
elif "qwen2.5-vl" in config.model.name.lower() or "olmocr-2-7b-1025" in config.model.name.lower():
    model_class = Qwen2_5_VLForConditionalGeneration
elif "qwen2-vl" in config.model.name.lower():
    model_class = Qwen2VLForConditionalGeneration

```

This conditional block ensures that any HuggingFace model identifier containing these substrings loads with the architecture-specific `*ForConditionalGeneration` class required for vision-language tasks.

## Configuring Models for SFT Training

You customize the training model by setting the `name` field within the `ModelConfig` section. The rest of the pipeline—including data loading, LoRA wrapping in [`olmocr/train/train.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/train.py), and checkpoint management in [`olmocr/train/prepare_checkpoint.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/prepare_checkpoint.py)—adapts automatically to the selected architecture.

### YAML Configuration Example

Configure a Qwen2.5-VL model with Flash Attention 2 using the following structure:

```yaml
model:
  name: "Qwen/Qwen2.5-VL-7B-Instruct"
  trust_remote_code: false
  load_in_8bit: false
  load_in_4bit: false
  device_map: auto
  torch_dtype: auto
  use_flash_attention: true
  attn_implementation: flash_attention_2
  use_lora: false

```

### Python Implementation

While the training script handles instantiation automatically, you can manually load any supported model using the corresponding Transformers class:

```python
from transformers import Qwen2VLForConditionalGeneration
import torch

# Example: Loading Qwen2-VL manually

model = Qwen2VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2-VL-7B-Instruct",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=False,
    attn_implementation="flash_attention_2",
)

```

## Key Files in the SFT Pipeline

Understanding these source files helps when extending the training system:

- **[`olmocr/train/config.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/config.py)** — Defines the `ModelConfig` dataclass (lines 177-184) where the model identifier and loading parameters are specified.
- **[`olmocr/train/train.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/train.py)** — Core training script implementing the model dispatch logic (lines 330-343) that selects the appropriate `*ForConditionalGeneration` class.
- **[`olmocr/train/dataloader.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/dataloader.py)** — Provides the data collator compatible with all supported vision-language models.
- **[`olmocr/train/prepare_checkpoint.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/prepare_checkpoint.py)** — Handles checkpoint serialization and loading for the selected model architecture.

## Summary

- **OlmOCR supports three model families** for SFT training: Qwen3-VL, Qwen2.5-VL (including the `olmocr-2-7b-1025` checkpoint), and Qwen2-VL.
- **Model selection occurs automatically** based on string matching in [`olmocr/train/train.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/train.py), dispatching to the correct HuggingFace `*ForConditionalGeneration` class.
- **The default model** is `Qwen/Qwen2.5-VL-7B-Instruct`, configured in [`olmocr/train/config.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/config.py).
- **Compatible identifiers** follow HuggingFace naming conventions and must contain the specific substrings (`qwen3-vl`, `qwen2.5-vl`, `qwen2-vl`, or `olmocr-2-7b-1025`) for the dispatch logic to recognize them.

## Frequently Asked Questions

### Can I use custom or fine-tuned Qwen models with olmOCR's SFT pipeline?

Yes. Any model identifier containing `qwen3-vl`, `qwen2.5-vl`, `qwen2-vl`, or `olmocr-2-7b-1025` will trigger the appropriate model class instantiation. Upload your custom checkpoint to HuggingFace and reference it in the `model.name` configuration field to use it for SFT training.

### Does the SFT training support LoRA for parameter-efficient fine-tuning?

Yes. The training pipeline supports **LoRA (Low-Rank Adaptation)** regardless of which Qwen VL model you select. Set `use_lora: true` in your `ModelConfig` to wrap the selected vision-language model with LoRA adapters before training begins.

### What happens if I specify a model that doesn't match the supported identifiers?

The training script will fail to identify a model class and will not proceed. The dispatch logic in [`olmocr/train/train.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/train.py) explicitly checks for the supported identifier fragments; any model name failing these checks will not instantiate a valid `*ForConditionalGeneration` class, preventing the SFT process from starting.

### Can I change the model architecture after starting a training run?

No, you must modify the configuration before launching the script. The model class is instantiated once at the beginning of training in [`olmocr/train/train.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/train.py). While you can resume training from checkpoints using the same architecture, switching to a different model family (e.g., from Qwen2-VL to Qwen3-VL) requires starting a new training session with the updated configuration.