Where to Find Documentation for olmOCR: Complete Guide to Installation, API, and Usage

The official documentation for olmOCR is located in the repository's README and the docs/source/ directory, specifically in overview.md and installation.md, which together provide comprehensive guidance on architecture, setup, and command-line usage.

The allenai/olmocr repository is an open-source OCR pipeline that runs large language model inference on PDFs and images to produce Dolma-compatible JSONL output. Whether you are installing the package for the first time or integrating the Python API into your own workflow, the documentation for olmOCR is maintained directly within the codebase through a combination of the top-level README and structured Sphinx-compatible markdown files.

Official Documentation Locations

The primary documentation for olmOCR is split between two locations in the repository:

README.md – The top-level README provides a high-level overview, quick-start commands, and basic installation instructions. This is the best starting point for new users.

docs/source/overview.md – This file contains the formal architectural overview, explaining how the pipeline orchestrates PDF processing, model inference, and document assembly.

docs/source/installation.md – Detailed installation instructions covering both pip install and source-based installation, including dependency requirements and environment setup.

These files are written in Markdown and can be browsed directly on GitHub or built into a Sphinx documentation site.

Core Architecture and Key Source Files

Understanding the documentation for olmOCR requires familiarity with the main entry points and utility modules. The pipeline architecture is implemented across the following key files:

• olmocr/pipeline.py – Contains the main() function, command-line argument parsing, and the primary worker orchestration logic. This file handles PDF fetching, page rendering, LLM inference via apost(), and document assembly through build_dolma_document().

• olmocr/work_queue.py – Implements the WorkQueue abstract class with concrete backends including LocalBackend and S3Backend for managing distributed processing jobs.

• olmocr/prompts/prompts.py – Houses the build_no_anchoring_v4_yaml_prompt() function and other prompt generation utilities used to format OCR requests for the language model.

• olmocr/prompts/anchor.py – Provides anchor-text extraction utilities for advanced prompting strategies.

• olmocr/data/renderpdf.py – Contains render_pdf_to_base64png(), which converts PDF pages to base64-encoded PNG images for model consumption.

• olmocr/filter/filter.py – Implements optional pre-processing filters for language detection and spam classification.

• olmocr/s3_utils.py – Utility functions including download_directory() and get_s3_bytes() for S3 interaction.

• olmocr/metrics.py – Defines MetricsKeeper for real-time token counting and performance tracking.

• olmocr/version.py – Centralizes the version string used in output metadata.

Installation Guide

According to the docs/source/installation.md file, you can install olmOCR via pip:

pip install olmocr

For development or contributions requiring the latest source code, clone the repository and install in editable mode:

git clone https://github.com/allenai/olmocr.git
cd olmocr
pip install -e .

Running the Pipeline

The primary documentation for running olmOCR is embedded in the CLI help and the README. The pipeline is executed via the module entry point defined in olmocr/pipeline.py:

python -m olmocr.pipeline \
  /tmp/olmocr-workspace \
  --pdfs s3://my-bucket/pdfs/*.pdf \
  --model allenai/olmOCR-2-7B-1025-FP8 \
  --workers 8 \
  --max_page_retries 6 \
  --markdown

Key parameters:

• /tmp/olmocr-workspace – Root directory for results and intermediate files.
• --pdfs – Glob pattern for source PDFs (supports S3 or local paths).
• --model – HuggingFace model repository, S3 path, or local directory containing the LLM.
• --markdown – Flag to generate .md files alongside the JSONL output.

Python API Reference

Beyond the command-line interface, the documentation for olmOCR includes several utility functions accessible via Python imports.

Rendering PDF Pages to Images

To convert a specific PDF page to a base64-encoded PNG (as used internally by the pipeline):

from olmocr.data.renderpdf import render_pdf_to_base64png

# Render page 1 with longest side of 1024px

png_b64 = render_pdf_to_base64png(
    "sample.pdf", 
    page=1, 
    target_longest_image_dim=1024
)

# Decode and save to file

import base64
with open("page1.png", "wb") as f:
    f.write(base64.b64decode(png_b64))

Source: olmocr/data/renderpdf.py

Generating OCR Prompts

Access the default "no-anchoring" YAML prompt used for LLM inference:

from olmocr.prompts import build_no_anchoring_v4_yaml_prompt

prompt_text = build_no_anchoring_v4_yaml_prompt()
print(prompt_text)

Source: olmocr/prompts/prompts.py

Calculating Output Paths

Determine where the markdown output will be written for a given input:

from olmocr.pipeline import get_markdown_path

workspace = "/tmp/olmocr-workspace"
source = "s3://my-bucket/pdfs/annual_report.pdf"
md_path = get_markdown_path(workspace, source)
print(md_path)  # Output: /tmp/olmocr-workspace/markdown/pdfs/annual_report.md

Source: olmocr/pipeline.py (lines 55-96)

Summary

• The complete documentation for olmOCR is located in the repository's README and the docs/source/ directory, with overview.md and installation.md serving as the primary references.
• The main entry point is olmocr/pipeline.py, which contains the main() function and CLI argument definitions.
• Key utilities for PDF rendering, prompt generation, and S3 operations are found in olmocr/data/renderpdf.py, olmocr/prompts/prompts.py, and olmocr/s3_utils.py respectively.
• Install via pip install olmocr or from source using the instructions in docs/source/installation.md.
• The pipeline supports both JSONL and Markdown output formats, configurable via command-line flags.

Frequently Asked Questions

Where is the official documentation for olmOCR hosted?

The official documentation is maintained within the allenai/olmocr GitHub repository itself. The README provides quick-start guidance, while the docs/source/ directory contains formal markdown files including overview.md for architecture details and installation.md for setup instructions.

How do I install olmOCR from source?

Clone the repository and install in editable mode using pip. The docs/source/installation.md file provides specific instructions for installing dependencies and setting up the development environment, including requirements for the vLLM server and optional CUDA dependencies.

What is the main entry point for the olmOCR pipeline?

The main entry point is the main() function located in olmocr/pipeline.py. This function parses command-line arguments, initializes the work queue (either local or S3-backed), starts the vLLM server if needed, and launches worker coroutines to process PDFs through the OCR pipeline.

How do I render a PDF page to an image using the olmOCR API?

Import render_pdf_to_base64png from olmocr.data.renderpdf. This function takes a PDF filepath, page number, and target dimension, returning a base64-encoded PNG string that can be decoded and saved or sent directly to the LLM inference endpoint.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}