# How to Generate Synthetic Training Data with OlmOCR: A Complete Pipeline Guide

> Learn to generate synthetic OCR training data using OlmOCR. This guide details the complete pipeline from PDF to JSONL with automated verification.

- Repository: [Ai2/olmocr](https://github.com/allenai/olmocr)
- Tags: how-to-guide
- Published: 2026-07-02

---

**OlmOCR generates synthetic OCR training data by converting PDF pages into HTML representations via the Claude API, transforming them into markdown with front-matter metadata, and exporting a JSONL file containing automated verification tests.**

The `allenai/olmocr` repository provides a specialized pipeline for creating high-quality synthetic training datasets for OCR models. By leveraging the Anthropic Claude API to interpret PDF layouts and generate structured HTML representations, the [`mine_html_templates.py`](https://github.com/allenai/olmocr/blob/main/mine_html_templates.py) script produces markdown ground truth paired with rigorous test suites. This guide walks through the complete workflow from raw PDFs to training-ready datasets.

## Prerequisites and Installation

Before generating synthetic data, install the core OlmOCR dependencies and Playwright browser automation tools.

```bash

# Install core dependencies

pip install -r requirements.txt

# Install Chromium for HTML rendering

playwright install chromium

```

The synthetic data generation requires access to the **Anthropic Claude API**. Export your API key as an environment variable before running any synthesis scripts:

```bash
export ANTHROPIC_API_KEY=sk-your-key-here

```

## Step 1: Prepare Your PDF Input Directory

Create a directory containing the PDFs you wish to convert into synthetic training examples. The pipeline will process every PDF in this location.

```bash
mkdir -p data/pdfs

# Copy your source PDFs into this directory

```

## Step 2: Execute the HTML Mining Pipeline

The **[`mine_html_templates.py`](https://github.com/allenai/olmocr/blob/main/mine_html_templates.py)** script in [`olmocr/synth/mine_html_templates.py`](https://github.com/allenai/olmocr/blob/main/olmocr/synth/mine_html_templates.py) drives the entire synthetic data generation process. This script orchestrates PDF rendering, HTML generation via Claude, markdown conversion, and test extraction.

```bash
python -m olmocr.synth.mine_html_templates \
    --input_dir data/pdfs \
    --output_dir synthetic_output \
    --name synthetic

```

### PDF to Image Rendering

First, the script converts each PDF page into a base64-encoded PNG using the **`render_pdf_to_base64png`** function defined in [`olmocr/data/renderpdf.py`](https://github.com/allenai/olmocr/blob/main/olmocr/data/renderpdf.py). These images serve as visual input for the Claude API.

### HTML Generation and Markdown Conversion

For each rendered page, the **`generate_html_from_image`** function sends the image to Claude with a specialized prompt requesting a full HTML representation that mirrors the original layout—including columns, headers, footers, images, mathematical notation, and tables.

The **`html_to_markdown_with_frontmatter`** function then processes this HTML using a custom **`PreserveTablesConverter`** class that ensures tables remain intact while converting the structure to markdown. This conversion preserves superscripts, subscripts, and tabular data that would otherwise be lost in standard HTML-to-text conversion.

### Test Suite Generation

The **`generate_tests_from_html`** function creates a comprehensive suite of automated tests including text-presence verification, reading-order validation, table structure checks, and rare-word detection. These tests ensure the synthetic data maintains fidelity to the source PDFs.

### Output Directory Structure

After execution, the `synthetic_output` directory contains:

- `bench_data/pdfs/synthetic/` – Symlinks to the original PDF files
- `bench_data/claude_original/synthetic/` – Generated markdown files with YAML front-matter
- `synthetic.jsonl` – JSONL file containing all test cases and metadata

## Step 3: Augment with Rotation (Optional)

To increase dataset diversity and improve model robustness, use **[`rotate_html_templates.py`](https://github.com/allenai/olmocr/blob/main/rotate_html_templates.py)** to apply rotation augmentation to a percentage of your synthetic data.

```bash
python -m olmocr.synth.rotate_html_templates \
    --input_dir synthetic_output \
    --output_dir synthetic_augmented \
    --rotation_percentage 5.0

```

This script copies the entire directory tree, rotates the selected PDFs using **pypdf**, and updates the corresponding front-matter fields to `is_rotation_valid: false` and `rotation_correction` with the applied rotation angle.

## Step 4: Load Data for Training

Feed the generated `synthetic.jsonl` directly into the OlmOCR training pipeline using the **`BaseMarkdownPDFDataset`** class located in [`olmocr/train/dataloader.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/dataloader.py).

```python
from olmocr.train.dataloader import BaseMarkdownPDFDataset
from transformers import AutoProcessor

processor = AutoProcessor.from_pretrained("Qwen2-VL-7B")

dataset = BaseMarkdownPDFDataset(
    root_dir="synthetic_output",
    pipeline_steps=processor,
)

print(f"Loaded {len(dataset)} synthetic training samples")

```

The dataset class reads the markdown files and associated test JSONL, tokenizing the content for model consumption. The **[`train.py`](https://github.com/allenai/olmocr/blob/main/train.py)** entry point in [`olmocr/train/train.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/train.py) orchestrates the full training loop and accepts any `BaseMarkdownPDFDataset` instance as input.

## Complete End-to-End Workflow

Combine all steps into a single automation pipeline:

```bash

# 1. Install dependencies

pip install -r requirements.txt
playwright install chromium

# 2. Configure API access

export ANTHROPIC_API_KEY=sk-...

# 3. Generate synthetic data

python -m olmocr.synth.mine_html_templates \
    --input_dir data/pdfs \
    --output_dir synthetic_output \
    --name synthetic

# 4. Apply rotation augmentation (optional)

python -m olmocr.synth.rotate_html_templates \
    --input_dir synthetic_output \
    --output_dir synthetic_augmented \
    --rotation_percentage 5

# 5. Begin training

python -m olmocr.train.train --config olmocr/train/configs/example_config.yaml

```

## Summary

- **Synthetic data generation** in OlmOCR relies on [`mine_html_templates.py`](https://github.com/allenai/olmocr/blob/main/mine_html_templates.py) to convert PDFs into HTML via the Claude API, then into markdown with front-matter.
- **Core utilities** include `render_pdf_to_base64png` in [`olmocr/data/renderpdf.py`](https://github.com/allenai/olmocr/blob/main/olmocr/data/renderpdf.py) for image generation and `PreserveTablesConverter` for maintaining table structure.
- **Automated testing** is handled by `generate_tests_from_html`, creating JSONL entries that validate text presence, ordering, and tabular integrity.
- **Data augmentation** via [`rotate_html_templates.py`](https://github.com/allenai/olmocr/blob/main/rotate_html_templates.py) adds rotational diversity by modifying a configurable percentage of PDFs.
- **Training integration** uses `BaseMarkdownPDFDataset` from [`olmocr/train/dataloader.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/dataloader.py) to consume the synthetic JSONL and markdown files.

## Frequently Asked Questions

### What file formats does OlmOCR require for synthetic data generation?

OlmOCR accepts standard **PDF files** as input for the synthetic generation pipeline. The [`mine_html_templates.py`](https://github.com/allenai/olmocr/blob/main/mine_html_templates.py) script processes these through Playwright's Chromium browser to generate PNG representations, then outputs **markdown files** with YAML front-matter and a **JSONL** file containing the test suite metadata.

### How does the pipeline handle complex document layouts like tables and mathematical notation?

The **`PreserveTablesConverter`** class in [`mine_html_templates.py`](https://github.com/allenai/olmocr/blob/main/mine_html_templates.py) specifically preserves table structures during HTML-to-markdown conversion, ensuring that rows, columns, and cell alignments remain intact. For mathematical notation, the Claude API generates HTML that includes superscripts and subscripts, which the converter maintains in the final markdown output alongside the visual layout information.

### Can I customize the percentage of documents receiving rotation augmentation?

Yes, the **[`rotate_html_templates.py`](https://github.com/allenai/olmocr/blob/main/rotate_html_templates.py)** script accepts a `--rotation_percentage` parameter that accepts float values between 0 and 100. The default is 5.0 percent, but you can adjust this based on your robustness requirements. This percentage determines what fraction of the PDFs are randomly selected for rotation using pypdf's rotation capabilities.

### What specific tests are included in the generated synthetic.jsonl file?

According to the implementation in [`mine_html_templates.py`](https://github.com/allenai/olmocr/blob/main/mine_html_templates.py), the **`generate_tests_from_html`** function creates several verification tests: **text-absence** checks to ensure no content was lost during conversion, **ordering** tests to verify reading sequence matches the original layout, **table** tests to validate row/column integrity, and **rare-word** tests to confirm unique vocabulary appears correctly in the output.