How to Convert Images to Text with olmOCR: A Complete Guide

You can convert images to text with olmOCR by passing image files (PNG/JPEG) to the --pdfs CLI flag, which automatically converts them to PDFs using convert_image_to_pdf_bytes and processes them through the VLLM pipeline to extract structured text.

The olmOCR library from AllenAI provides a robust pipeline for extracting structured text from documents. While designed primarily for PDFs, it handles raster images seamlessly by converting them to PDF format before processing. This guide explains exactly how to convert images to text with olmOCR using both command-line and programmatic approaches, referencing the actual implementation in the allenai/olmocr repository.

How olmOCR Processes Image Files

olmOCR treats images as first-class inputs by transparently converting them to PDF documents before running the OCR model. When you provide an image file path, the pipeline detects the format and invokes specialized conversion logic to create a temporary PDF in memory.

Image-to-PDF Conversion Internals

The core conversion happens in olmocr/image_utils.py (lines 6-34) within the convert_image_to_pdf_bytes function. This utility validates that the input file exists, then calls the external img2pdf binary via subprocess.run (lines 20-38), capturing the stdout to return raw PDF bytes. This approach supports PNG, JPEG, and other common raster formats without requiring additional image processing libraries.

Pipeline Detection and Routing

In olmocr/pipeline.py (lines 585-595), the process_pdf function checks whether the input is an image using the is_png and is_jpeg helper functions. If the file is detected as a raster image, the pipeline rewrites the temporary file with the PDF bytes returned from convert_image_to_pdf_bytes, allowing the rest of the system to treat it as a standard PDF document. From there, build_page_query (lines 10-42) renders the page as a base64-encoded PNG and constructs the VLLM request payload.

Command-Line Usage to Convert Images to Text

The simplest way to convert images to text with olmOCR is through the command-line interface. Ensure you have the img2pdf binary installed and available on your system PATH before running the pipeline.

# Install the package and dependencies

pip install olmocr

# Run the pipeline on one or many images

python -m olmocr.pipeline \
    s3://my-bucket/workspace \
    --pdfs image1.png image2.jpg \
    --model allenai/olmOCR-2-7B-1025-FP8 \
    --markdown

The --pdfs flag accepts any file path, and the pipeline automatically detects PNG/JPEG extensions to trigger the conversion. The --markdown flag writes the extracted text to *.md files alongside the JSONL output.

Programmatic Image-to-Text Conversion

For custom workflows, you can invoke the conversion functions directly from Python. This is useful when integrating olmOCR into larger applications or when processing images in memory without writing intermediate files to disk.

from pathlib import Path
from olmocr.image_utils import convert_image_to_pdf_bytes
from olmocr.pipeline import build_page_query, render_pdf_to_base64png
import asyncio
import base64

async def image_to_text(image_path: Path, target_dim: int = 2048) -> str:
    # 1. Convert image to PDF bytes using img2pdf

    pdf_bytes = convert_image_to_pdf_bytes(str(image_path))
    
    # 2. Write temporary PDF to disk (required by render function)

    tmp_pdf = Path("/tmp/temp_image.pdf")
    tmp_pdf.write_bytes(pdf_bytes)
    
    try:
        # 3. Render the first page to base64 PNG for the VLLM model

        img_b64 = await asyncio.to_thread(
            render_pdf_to_base64_png, 
            str(tmp_pdf), 
            page=1, 
            target_longest_image_dim=target_dim
        )
        
        # 4. Build the query payload for the model

        query = await build_page_query(
            local_pdf_path=str(tmp_pdf),
            page=1,
            target_longest_image_dim=target_dim,
            model_name="olmocr",
        )
        
        # The query contains the base64 image in:

        # query["messages"][0]["content"][1]["image_url"]["url"]

        
        # 5. Send query to your VLLM endpoint here

        # response = await httpx.AsyncClient().post(...)

        
        return "Extracted text would appear here from response"
    finally:
        tmp_pdf.unlink()

# Run the conversion

result = asyncio.run(image_to_text(Path("sample.jpg")))

Note that render_pdf_to_base64png and build_page_query in olmocr/pipeline.py (lines 10-42) handle the image preparation and request construction, converting the temporary PDF page into a format suitable for the vision-language model.

Accessing the Extracted Text

Once processing completes, the extracted text resides in the natural_text attribute of the PageResponse object. When using the --markdown flag, the pipeline writes plain text files to stable paths derived from the source filename, while JSONL output contains the text in the text field of each Dolma document.

import json

# Reading from markdown output

with open("workspace/markdown/path/to/image1.md") as f:
    plain_text = f.read()

# Or reading from JSONL output

with open("workspace/results.jsonl") as f:
    for line in f:
        doc = json.loads(line)
        extracted_text = doc["text"]

Summary

• olmOCR converts images to PDFs internally: The convert_image_to_pdf_bytes function in olmocr/image_utils.py uses the img2pdf binary to transform PNG/JPEG files into PDF bytes before processing.
• Automatic format detection: The pipeline in olmocr/pipeline.py (lines 585-595) automatically detects image formats and routes them through the conversion step when you pass files to --pdfs.
• Flexible output formats: Use --markdown to generate clean .md text files, or parse the JSONL output to access the natural_text field from PageResponse objects.
• Programmatic access: You can import convert_image_to_pdf_bytes directly to handle image-to-PDF conversion in custom Python workflows before calling the rendering and query building functions.

Frequently Asked Questions

Does olmOCR support multiple image formats?

Yes, olmOCR supports any format that the img2pdf binary can process, including PNG, JPEG, and other common raster formats. The is_png and is_jpeg helpers in olmocr/image_utils.py handle the initial detection, while convert_image_to_pdf_bytes manages the actual conversion regardless of the specific image type.

Do I need to install img2pdf separately?

Yes, you must install the img2pdf binary and ensure it is available on your system PATH. The convert_image_to_pdf_bytes function in olmocr/image_utils.py (lines 20-38) calls this binary via subprocess.run, so the pipeline will fail if the dependency is missing.

Can I process images without writing temporary PDF files to disk?

While the current pipeline implementation in olmocr/pipeline.py writes the PDF bytes to a temporary file before rendering (as seen in lines 585-595), you can work around this in custom code by using convert_image_to_pdf_bytes to get the PDF data in memory, then integrating directly with the rendering logic if you modify the pipeline to accept byte streams instead of file paths.

How do I get plain text output instead of JSON?

Pass the --markdown flag when running the pipeline. This triggers the Markdown writer logic in olmocr/pipeline.py (lines 66-94), which creates {source}.md files containing only the extracted natural_text without the JSON structure. Alternatively, extract the text field from each line of the JSONL output file.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}