OlmOCR Temperature Selection Algorithm: How Dynamic Retry Temperatures Improve OCR Output Quality

OlmOCR implements a progressive temperature selection algorithm that starts at 0.1 for deterministic initial attempts and scales up to 1.0 across eight retry attempts to overcome repetition errors and maximize output quality.

The allenai/olmocr repository uses an intelligent temperature selection algorithm to dynamically adjust sampling randomness during PDF processing. This temperature-by-attempt strategy balances deterministic output with creative diversity, automatically trading precision for exploration when the model encounters repetitive generation loops.

How the Temperature Selection Algorithm Works in OlmOCR

The temperature selection algorithm operates as a deterministic retry policy hardcoded into the inference pipeline. Rather than using a static temperature value, OlmOCR maps each retry attempt to a specific temperature drawn from a predefined schedule.

The TEMPERATURE_BY_ATTEMPT Schedule

In olmocr/pipeline.py, the algorithm defines a hardcoded list named TEMPERATURE_BY_ATTEMPT that specifies temperature values for successive retry attempts:

# olmocr/pipeline.py

# Temperature values for retry attempts – higher temperature helps overcome repetition issues

TEMPERATURE_BY_ATTEMPT = [0.1, 0.1, 0.2, 0.3, 0.5, 0.8, 0.9, 1.0]

Source: pipeline.py L84-L86

This schedule provides eight distinct temperature steps, ranging from highly deterministic (0.1) to maximum diversity (1.0).

Mapping Attempt Index to Temperature Values

When processing a page, the pipeline determines the appropriate temperature by capping the current attempt index to the length of the schedule:

temp_idx = min(attempt, len(TEMPERATURE_BY_ATTEMPT) - 1)
temperature = TEMPERATURE_BY_ATTEMPT[temp_idx]

Source: pipeline.py L64-L66

The attempt parameter starts at 0, meaning the first two retries both use 0.1, while subsequent attempts progressively increase randomness up to the eighth attempt.

Injecting Temperature into the LLM Query

The pipeline constructs an initial query dictionary with a placeholder temperature of 0.0, then overwrites this value with the selected temperature immediately before sending the HTTP request:

query = await build_page_query(...)
query["temperature"] = temperature

Source: pipeline.py L71-L78

This injection occurs after the build_page_query function prepares the payload, ensuring the temperature selection algorithm has the final say on sampling parameters for each retry.

Why Progressive Temperature Scaling Improves Output Quality

The temperature selection algorithm improves OCR quality through a deliberate tradeoff between determinism and diversity:

• Low temperatures (0.1-0.2): Restrict the model to high-probability token sequences, producing consistent, predictable output ideal for clean documents on initial attempts.
• Moderate temperatures (0.3-0.5): Introduce slight variation to escape minor repetition loops without sacrificing accuracy.
• High temperatures (0.8-1.0): Maximize token diversity to break out of persistent repetitive generation patterns that cause validation failures.

By starting with deterministic sampling and escalating to exploratory sampling only when retries are necessary, OlmOCR maintains high accuracy for straightforward documents while retaining the flexibility to handle edge cases that trigger repetitive outputs.

Implementing the Temperature Logic in Your Own Code

You can emulate OlmOCR's temperature selection algorithm using the same logic found in the pipeline:

# Example: manually emulate the retry logic for a single page

from olmocr.pipeline import TEMPERATURE_BY_ATTEMPT

def get_temperature(attempt: int) -> float:
    """Return the temperature that the pipeline would use for a given attempt."""
    idx = min(attempt, len(TEMPERATURE_BY_ATTEMPT) - 1)
    return TEMPERATURE_BY_ATTEMPT[idx]

for attempt in range(6):
    print(f"Attempt {attempt}: temperature = {get_temperature(attempt)}")

Output:

Attempt 0: temperature = 0.1
Attempt 1: temperature = 0.1
Attempt 2: temperature = 0.2
Attempt 3: temperature = 0.3
Attempt 4: temperature = 0.5
Attempt 5: temperature = 0.8

To integrate this into custom inference code, inject the temperature into your query payload as OlmOCR does:

# Example: how the pipeline injects the temperature into the request payload

async def example_query(pdf_path, page, attempt, args):
    query = await build_page_query(pdf_path, page, args.target_longest_image_dim,
                                  model_name=args.model)
    query["temperature"] = get_temperature(attempt)   # same logic as pipeline

    # send query to LLM server …

Summary

• Progressive temperature schedule: OlmOCR uses TEMPERATURE_BY_ATTEMPT = [0.1, 0.1, 0.2, 0.3, 0.5, 0.8, 0.9, 1.0] to map retry attempts to specific temperature values.
• Deterministic to diverse: The algorithm starts with low temperatures for accuracy and scales to high temperatures to break repetition loops.
• Pipeline integration: The temperature is injected into the query dictionary in olmocr/pipeline.py immediately before the LLM API call.
• Automatic optimization: This lightweight retry policy requires no manual tuning while improving OCR success rates across varied document types.

Frequently Asked Questions

What is the temperature selection algorithm in OlmOCR?

The temperature selection algorithm in OlmOCR is a deterministic retry policy that assigns specific temperature values to each retry attempt when processing PDF pages. Defined in olmocr/pipeline.py, it uses a hardcoded list called TEMPERATURE_BY_ATTEMPT to progressively increase sampling randomness from 0.1 to 1.0 as retries accumulate.

How does OlmOCR handle temperature on retry attempts?

OlmOCR handles temperature by indexing into the TEMPERATURE_BY_ATTEMPT list using min(attempt, len(TEMPERATURE_BY_ATTEMPT) - 1), ensuring the first two attempts use 0.1, the third uses 0.2, and so on up to 1.0. This value overwrites the placeholder temperature in the query dictionary before each LLM API request.

Why does OlmOCR use low temperatures for initial attempts?

OlmOCR uses low temperatures (0.1) for initial attempts because deterministic sampling favors the most statistically likely token sequences, producing consistent and accurate OCR results for clean documents. Low temperatures minimize hallucination and maintain high precision when the model correctly interprets the page on the first try.

Can I customize the temperature values in OlmOCR?

While the TEMPERATURE_BY_ATTEMPT list is hardcoded in olmocr/pipeline.py, you can customize temperature values by modifying the source code or by implementing the get_temperature logic in your own wrapper code. For one-shot runs, the training CLI in olmocr/train/grpo_train.py exposes a --temperature flag that defaults to 0.8.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}