What Input Formats Does olmocr Support? Complete Guide to Document Processing

olmocr supports PDF documents, PNG and JPEG images, gzip-compressed tarballs of PDFs, and plain-text manifest files containing lists of input paths.

olmocr is an open-source document conversion pipeline developed by Allen AI that transforms scanned documents and images into structured Markdown. Understanding which input formats the tool accepts is essential for batch processing academic papers, archives, and image-based documents, as implemented in the allenai/olmocr repository.

Supported Document Formats

PDF Documents

olmocr processes PDF files by verifying both the .pdf extension and the %PDF header signature at the start of the file. According to lines 1351‑1352 in olmocr/pipeline.py, this dual validation ensures only valid PDF documents enter the conversion pipeline, preventing corrupted or misnamed files from causing downstream errors.

PNG and JPEG Images

The pipeline supports PNG and JPEG/JPG images through extension-based detection and header validation functions. For PNG files, the code invokes is_png() (lines 1352‑1353), while JPEG validation uses is_jpeg() (lines 1353‑1354). The README confirms these formats at lines 30‑33, stating the tool converts "PDF, PNG, and JPEG based documents into clean Markdown."

Batch Processing and Archives

Tarball Archives of PDFs

For large-scale processing, olmocr accepts .tar.gz or .tgz archives containing multiple PDFs. The is_tarball_path function defined at line 479 in olmocr/pipeline.py identifies these archives, which are unpacked and processed as a single work item, allowing you to submit entire datasets as one input.

Path List Files

When processing thousands of documents, you can supply a plain-text file with the .txt extension containing one absolute or relative path per line. As implemented in lines 64‑69 of olmocr/pipeline.py, this manifest approach avoids shell argument length limitations and enables you to queue mixed collections of PDFs, images, and tarballs.

Command-Line Usage Examples

The following commands demonstrate how to process each supported format using the olmocr CLI:

# Convert a single PDF with GPU acceleration

olmocr ./myworkspace --markdown --pdfs mydoc.pdf

# Convert PNG and JPEG images (note: uses --pdfs flag)

olmocr ./myworkspace --markdown --pdfs figure.png
olmocr ./myworkspace --markdown --pdfs photo.jpg

# Process multiple PDFs using glob patterns

olmocr ./myworkspace --markdown --pdfs tests/gnarly_pdfs/*.pdf

# Process a tarball of PDFs

olmocr ./myworkspace --markdown --pdfs archive_of_pdfs.tar.gz

# Use a text file listing input paths (one per line)

olmocr ./myworkspace --markdown --pdfs inputs.txt

All commands route through the file-type detection logic in olmocr/pipeline.py, which instantiates the appropriate reader (PdfReader, is_png, is_jpeg) for each detected format.

Summary

• PDF documents are validated using extension checks and %PDF header verification in olmocr/pipeline.py lines 1351‑1352.
• PNG and JPEG images are supported via is_png() and is_jpeg() validation functions at lines 1352‑1354.
• Tarball archives (.tar.gz, .tgz) are automatically identified by is_tarball_path at line 479 and unpacked for processing.
• Path list files (.txt) enable batch processing of large document collections without command-line constraints, as handled at lines 64‑69.

Frequently Asked Questions

Does olmocr support TIFF or BMP images?

No. According to the source code in olmocr/pipeline.py, only PNG and JPEG images are validated through the is_png() and is_jpeg() functions. The README explicitly lists only PDF, PNG, and JPEG as supported formats for conversion.

Can I mix different file types in a single command?

Yes. You can specify PDFs, images, and tarballs in the same command or path list file. The pipeline processes each file according to its detected type, routing images and PDFs through their respective loaders while unpacking any tarballs encountered.

How does olmocr handle corrupted PDF files?

Corrupted files are rejected during the validation phase. At lines 1351‑1352 in olmocr/pipeline.py, the code checks for the %PDF magic number after confirming the file extension. Files lacking this header signature are skipped before processing begins.

Is there a file size limit for tarball archives?

The analysis does not specify a hard size limit for tarballs. The is_tarball_path function at line 479 identifies archives by extension, and the pipeline unpacks them for processing. System memory and available disk space for the workspace directory are the practical limiting factors.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}