# Understanding the Temperature Retry Mechanism in OlmOCR: Handling Model Failures

> Learn how OlmOCR's temperature retry mechanism handles model failures by increasing LLM sampling temperature across eight attempts, improving generation and recovery.

- Repository: [Ai2/olmocr](https://github.com/allenai/olmocr)
- Tags: internals
- Published: 2026-07-05

---

**OlmOCR uses a temperature-based retry mechanism that progressively increases the LLM's sampling temperature from 0.1 to 1.0 across eight attempts to escape repetitive generation patterns and recover from model failures.**

The `allenai/olmocr` pipeline implements a sophisticated *temperature retry mechanism* to ensure robust OCR processing when large language models encounter generation errors or repetitive outputs. By gradually escalating the randomness of model sampling, this system provides multiple recovery paths before falling back to deterministic OCR engines.

## How the Temperature Retry Mechanism Works

The mechanism relies on a predefined schedule in [`olmocr/pipeline.py`](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py) that maps each retry attempt to a specific temperature value, balancing determinism with recovery potential.

### The Temperature Schedule

The static list `TEMPERATURE_BY_ATTEMPT` defines the progression:

```python

# olmocr/pipeline.py

TEMPERATURE_BY_ATTEMPT = [0.1, 0.1, 0.2, 0.3, 0.5, 0.8, 0.9, 1.0]

```

This configuration allows two initial attempts at very low temperature (0.1), followed by gradual escalation to maximum randomness (1.0) on the eighth attempt. Lower temperatures prioritize deterministic outputs for well-behaved pages, while higher temperatures inject diversity to break error loops.

### Temperature Assignment Logic

The `try_single_page` function selects the appropriate temperature based on the current attempt counter:

```python

# olmocr/pipeline.py

temp_idx = min(attempt, len(TEMPERATURE_BY_ATTEMPT) - 1)
temperature = TEMPERATURE_BY_ATTEMPT[temp_idx]
query["temperature"] = temperature

```

The code clamps the attempt index to prevent array overflows while ensuring each retry receives progressively higher temperature values in the LLM query payload sent to `/chat/completions`.

### The Retry Loop Orchestration

The `process_page` function manages the retry lifecycle:

```python

# olmocr/pipeline.py

MAX_RETRIES = args.max_page_retries
retry_attempts = list(range(1, MAX_RETRIES))

# First attempt (attempt 0) uses temperature 0.1

# Subsequent failures trigger retries with escalating temperatures

result = await try_single_page_with_backoff(..., attempt, cumulative_rotation)

```

The first attempt executes with the lowest temperature (0.1). Upon validation failure, the loop advances to the next attempt, automatically selecting higher temperatures according to the schedule until either validation passes or the list exhausts.

## Handling Different Failure Types

The mechanism distinguishes between transient network issues and substantive model generation failures, applying different strategies for each.

### HTTP and Content Validation Failures

When the remote server returns a non-200 status code, `try_single_page` returns `None`, triggering the temperature escalation sequence. For successful HTTP responses, the pipeline validates:
- `total_tokens` ≤ `MODEL_MAX_CONTEXT`
- `finish_reason == "stop"`

If either check fails, the system marks the result invalid and proceeds to the next temperature tier, allowing the model to generate alternative responses under different sampling conditions.

### Connection Errors and Backoff Strategy

Network interruptions are handled separately through `try_single_page_with_backoff`, which wraps the core function with exponential backoff timing (10 seconds × 2^n). These backoff retries are independent of the temperature schedule—they wait for network recovery while maintaining the same temperature value, preserving the escalation strategy for generation failures only.

## Fallback Strategy When Retries Exhaust

If all eight temperature retries fail to produce valid output, the pipeline invokes `make_fallback_result` to ensure document completion:

```python

# olmocr/pipeline.py

return make_fallback_result(pdf_orig_path, pdf_local_path, page_num)

```

This function switches to the `pdftotext` OCR engine, guaranteeing that LLM failures never block the overall processing workflow. The fallback acts as a safety net when the temperature retry mechanism cannot resolve model-specific generation errors.

## Practical Implementation Example

The following example demonstrates how to exercise the retry logic for a single page:

```python
import asyncio
from olmocr.pipeline import try_single_page, try_single_page_with_backoff, TEMPERATURE_BY_ATTEMPT

async def demo_retry(args, pdf_path, page):
    for attempt in range(len(TEMPERATURE_BY_ATTEMPT)):
        result = await try_single_page_with_backoff(
            args,
            pdf_orig_path=pdf_path,
            pdf_local_path=pdf_path,
            page_num=page,
            attempt=attempt,
            rotation=0,
        )
        if result and result.is_valid and result.response.is_rotation_valid:
            print(f"Success on attempt {attempt} (temp={TEMPERATURE_BY_ATTEMPT[attempt]})")
            break
        else:
            print(f"Attempt {attempt} failed – retrying with higher temperature")
    else:
        print("All retries exhausted – fallback will be used")

```

## Summary

- **Progressive temperature escalation**: The `TEMPERATURE_BY_ATTEMPT` array in [`olmocr/pipeline.py`](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py) starts at 0.1 (near-deterministic) and scales to 1.0 (maximum randomness) across eight attempts.
- **Separation of concerns**: Network failures trigger exponential backoff without temperature changes, while generation failures trigger temperature increases via `try_single_page`.
- **Deterministic fallback**: The `make_fallback_result` function ensures processing completion via `pdftotext` when all LLM retries exhaust, preventing pipeline stalls.

## Frequently Asked Questions

### Why does OlmOCR use temperature escalation instead of fixed retries?

Fixing a low temperature can trap the model in repetitive error patterns when it encounters difficult layouts or ambiguous text. By gradually increasing temperature from 0.1 to 1.0, the system allows the LLM to explore alternative phrasings and escape local minima in the probability distribution without sacrificing determinism on easier pages.

### How many retry attempts does OlmOCR allow by default?

The default configuration supports up to eight attempts, defined by the eight values in `TEMPERATURE_BY_ATTEMPT`. The `MAX_RETRIES` variable is set from `args.max_page_retries`, allowing customization while being clamped to the length of the temperature schedule to prevent index errors.

### Does raising the temperature guarantee successful OCR?

No. Higher temperatures increase output diversity but do not guarantee validity. If all eight temperature levels fail validation checks—such as exceeding `MODEL_MAX_CONTEXT` or returning a non-stop finish reason—the system falls back to `pdftotext` to ensure the pipeline delivers a result rather than failing silently.

### Where is the temperature retry mechanism implemented in the codebase?

The core logic resides in [`olmocr/pipeline.py`](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py), specifically within the `TEMPERATURE_BY_ATTEMPT` schedule (lines 84-87), the `try_single_page` function that assigns temperatures to queries, and the `process_page` function that orchestrates the retry loop and validation logic.