# How Does OlmOCR Process OCR Tasks: An Eight-Stage Technical Pipeline

> Explore OlmOCR's eight-stage pipeline for processing OCR tasks. Discover how it converts PDFs to structured documents using LLMs, rotation correction, and retry logic.

- Repository: [Ai2/olmocr](https://github.com/allenai/olmocr)
- Tags: deep-dive
- Published: 2026-07-07

---

**OlmOCR converts PDFs into structured Dolma documents by rendering each page as a base64-encoded PNG image and processing it through a vision-capable large language model, featuring automatic rotation correction, intelligent retry logic, and a pdftotext fallback mechanism.**

The AllenAI OlmOCR repository implements a sophisticated async pipeline that transforms static PDF documents into machine-readable structured data. Understanding how OlmOCR processes OCR tasks requires examining its eight-stage architecture, which orchestrates PDF ingestion, vision-language model inference, and robust error recovery. The system outputs standardized Dolma JSON-L documents with rich per-page metadata stored in document attributes.

## The Eight-Stage OCR Pipeline

### Stage 1: PDF Loading and Ingestion

The pipeline begins in [`olmocr/pipeline.py`](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py) where the `process_pdf` function (line 75) handles document acquisition. The system reads input from S3 buckets, local disk, or tarballs, converting JPEG or PNG images to PDF format when necessary. This stage establishes the async processing context and prepares the temporary workspace for downstream operations.

### Stage 2: Page-by-Page Rendering

Within `build_page_query` (line 6), each page renders to a base64-encoded PNG with a configurable maximum longest dimension. The implementation uses a bounded async semaphore to prevent resource exhaustion, ensuring the host never spawns excessive concurrent workers. This rendering transforms document pages into the image format required by the vision-capable LLM.

### Stage 3: LLM Request Construction and Inference

The `try_single_page` function (line 61) constructs the inference request sent to the chat completions endpoint. The payload includes the PNG image data alongside the fixed YAML prompt generated by `build_no_anchoring_v4_yaml_prompt` from [`olmocr/prompts.py`](https://github.com/allenai/olmocr/blob/main/olmocr/prompts.py). The system implements adaptive retry logic that increases temperature on each retry to break looping outputs, maximizing successful extraction rates.

### Stage 4: Response Parsing and Front Matter Extraction

After receiving the model's markdown response, the `FrontMatterParser` (defined in [`olmocr/train/front_matter.py`](https://github.com/allenai/olmocr/blob/main/olmocr/train/front_matter.py)) extracts structured metadata at line 110 of the pipeline. This component parses the markdown front matter containing language detection, rotation flags, table presence, and diagram indicators into a typed `PageResponse` object, enabling programmatic access to page-level attributes.

### Stage 5: Rotation Correction and Retry Logic

The `process_page` function (line 84) handles geometric corrections when the model indicates `is_rotation_valid=False`. The pipeline tracks `cumulative_rotation` across sequential attempts, rotating the page image and retrying inference up to the configurable `args.max_page_retries` limit. Non-rotation errors trigger parallel retry attempts once the VLLM request queue empties, optimizing throughput during error recovery.

### Stage 6: OCR Fallback Mechanism

When the LLM exhausts all retry attempts, `make_fallback_result` (line 33) executes a pure-OCR fallback using `pdftotext`. This mechanism ensures text recovery even when the vision model fails to produce valid output, guaranteeing document accessibility regardless of image quality or layout complexity.

### Stage 7: Dolma Document Assembly

The `build_dolma_document` function (line 202) aggregates all `PageResult` objects into a single structured document. The concatenated natural text forms the document body, while per-page metadata—including language codes, rotation angles, and structural flags—populates the `attributes` dictionary. This format aligns with the Dolma corpus specification for downstream NLP training workflows.

### Stage 8: Output Generation and Persistence

Finally, the `worker` function (line 98) persists the completed document to the workspace or S3 as JSON-L. When the `--markdown` flag is enabled, the pipeline also emits a plain-text `.md` version alongside the structured output, accommodating both human review and machine processing requirements.

## Distributed Processing Architecture

Beyond single-document processing, the `WorkQueue` class in [`olmocr/work_queue.py`](https://github.com/allenai/olmocr/blob/main/olmocr/work_queue.py) orchestrates distributed batch operations across multiple workers. This component handles S3-based locking mechanisms and local lock files to prevent duplicate processing in distributed environments. The `MetricsKeeper` and `WorkerTracker` utilities in [`olmocr/metrics.py`](https://github.com/allenai/olmocr/blob/main/olmocr/metrics.py) maintain live token counts and success-rate statistics throughout the pipeline execution.

## Running OlmOCR Locally

To process a single PDF through the pipeline, initialize the requisite arguments and invoke `process_pdf` asynchronously:

```python
import asyncio
from argparse import Namespace
from olmocr.pipeline import process_pdf

args = Namespace(
    server="http://localhost:8000",
    api_key=None,
    model="olmocr",
    guided_decoding=False,
    target_longest_image_dim=1024,
    max_page_retries=5,
    max_page_error_rate=0.2,
    apply_filter=False,
    markdown=False,
    workspace="/tmp/olmocr_workspace",
)

async def main():
    doc = await process_pdf(args, worker_id=0, pdf_orig_path="s3://my-bucket/sample.pdf")
    print(doc)  # Dolma JSON-L document

asyncio.run(main())

```

For batch processing workloads, leverage the `WorkQueue` with multiple concurrent workers:

```python
import asyncio
from argparse import Namespace
from olmocr.work_queue import WorkQueue, LocalBackend
from olmocr.pipeline import worker

args = Namespace(
    server="http://localhost:8000",
    api_key=None,
    model="olmocr",
    guided_decoding=False,
    target_longest_image_dim=1024,
    max_page_retries=5,
    max_page_error_rate=0.2,
    apply_filter=False,
    markdown=True,
    workspace="/tmp/olmocr_workspace",
    max_workers=4,
)

async def run():
    backend = LocalBackend(workspace_path=args.workspace)
    queue = WorkQueue(backend)
    
    await queue.populate_queue(
        work_paths=["data/file1.pdf", "data/file2.pdf"],
        items_per_group=5
    )
    await queue.initialize_queue()
    
    workers = [
        asyncio.create_task(worker(args, queue, worker_id=i))
        for i in range(args.max_workers)
    ]
    await asyncio.gather(*workers)

asyncio.run(run())

```

## Summary

- **Eight-stage pipeline**: OlmOCR processes PDFs through ingestion, rendering, LLM inference, parsing, rotation correction, fallback recovery, assembly, and output stages.
- **Vision-first approach**: Pages convert to PNG images before processing through the `build_no_anchoring_v4_yaml_prompt` via chat completions endpoints.
- **Robust error handling**: The system implements bounded async semaphores, temperature-based retry logic, and `pdftotext` fallbacks to maximize extraction success.
- **Structured output**: Results conform to the Dolma document specification with rich metadata attributes captured from front-matter parsing.
- **Distributed ready**: The `WorkQueue` architecture supports S3-backed distributed processing with locking mechanisms to prevent duplicate work.

## Frequently Asked Questions

### What happens when the LLM fails to extract text from a page?

The pipeline implements a multi-layered recovery system. First, `process_page` retries the request with rotation corrections if the model flags orientation issues. After exhausting `max_page_retries` attempts, `make_fallback_result` invokes `pdftotext` to extract raw text without vision model processing. This ensures every page yields usable output even when image quality or layout complexity challenges the LLM.

### How does OlmOCR handle incorrectly rotated pages?

During the `try_single_page` execution, the model returns `is_rotation_valid` status in the front matter. When this flag is false, the `process_page` function increments the `cumulative_rotation` counter and re-renders the page at the corrected angle before resubmitting to the LLM. This retry loop continues until the model validates the orientation or the system reaches the configured retry limit.

### Can OlmOCR process image files directly, or only PDFs?

The `process_pdf` function automatically converts JPEG and PNG inputs to PDF format before processing. This conversion happens during the ingestion stage, allowing the pipeline to treat all visual inputs uniformly. The system handles the format transformation transparently, requiring no manual intervention from users working with scanned image documents.

### What is the Dolma document format produced by OlmOCR?

The `build_dolma_document` function generates a JSON-L structured document where the `text` field contains concatenated page content and the `attributes` dictionary stores per-page metadata. This metadata includes detected language, rotation angles, table presence flags, and diagram indicators extracted by the `FrontMatterParser`. The format follows the Dolma corpus specification designed for large-scale NLP training datasets.