What Programming Languages Are Used in AllenAI/olmocr?

The allenai/olmocr repository is primarily written in Python, with Bash scripts handling training orchestration and Dockerfiles providing containerized environments.

The allenai/olmocr project is an open-source OCR pipeline designed for processing PDF documents at scale. Understanding what programming languages are used in allenai/olmocr reveals a Python-centric architecture supplemented by shell automation and container definitions. This multilingual structure supports both rapid development and production deployment workflows across diverse computing environments.

Primary Programming Language: Python

Core Library and Pipelines

Python serves as the dominant programming language in allenai/olmocr, powering the central OCR engine, data-processing pipelines, and utility modules. The primary implementation resides in olmocr/pipeline.py, which contains the core Pipeline class responsible for document ingestion and text extraction.

from olmocr.pipeline import Pipeline

# Initialise the pipeline with default configuration

pipeline = Pipeline()

# Run OCR on a PDF file

output = pipeline.run("example.pdf")
print(output["text"])

Viewer Templates

The repository also includes HTML templates utilized by Python viewer components. Specifically, olmocr/viewer/dolmaviewer_template.html provides the frontend markup rendered by the Python-based viewer application, demonstrating how the project leverages web technologies within its Python framework.

Orchestration Layer: Bash Shell Scripts

Bash scripts provide the automation layer for training, benchmarking, and data-generation workflows. These shell scripts orchestrate complex multi-step operations that invoke Python modules with specific environment configurations.

The scripts/train/newtrainer-frontier.sh script exemplifies this pattern, handling distributed training launches on GPU clusters:

#!/usr/bin/env bash

# Run the new trainer on the frontier GPU cluster

scripts/train/newtrainer-frontier.sh \
    --model gpt-4o-mini \
    --data_path data/olmocrmix \
    --output_dir results/

Additionally, scripts/run_benchmark.sh wraps benchmark execution suites, enabling consistent performance evaluation across different hardware configurations.

Containerization: Dockerfile Syntax

The project utilizes Dockerfile syntax to define reproducible execution environments. The root Dockerfile specifies a Python 3.11-slim base image and manages system dependencies critical for PDF processing:

FROM python:3.11-slim

# Install system dependencies

RUN apt-get update && apt-get install -y git && rm -rf /var/lib/apt/lists/*

# Install the Python package

COPY . /app
WORKDIR /app
RUN pip install -e .

This container definition ensures that the Python-based OCR pipeline executes consistently across development machines and production clusters.

Configuration and Metadata

While not programming languages themselves, the repository includes essential configuration files that support the primary languages. The pyproject.toml file manages Python packaging metadata and dependency resolution, while .github/workflows/main.yml defines CI/CD automation in YAML syntax. These files complement the core codebase but do not constitute additional programming languages.

Summary

• Python forms the core library, OCR pipeline logic, and utility implementations in files like olmocr/pipeline.py.
• Bash scripts automate training workflows via scripts/train/newtrainer-frontier.sh and benchmarking via scripts/run_benchmark.sh.
• Dockerfile syntax defines containerized environments in the root Dockerfile for reproducible deployment.
• HTML templates in olmocr/viewer/dolmaviewer_template.html support the Python viewer components.

Frequently Asked Questions

Is allenai/olmocr written entirely in Python?

No. While the core OCR functionality and data pipelines are implemented in Python, the repository relies on Bash scripts for workflow orchestration and Dockerfiles for environment containerization. According to the source code analysis, there are no compiled languages like C++ or JavaScript applications present.

What are the Bash scripts used for in olmocr?

The Bash scripts handle automation tasks that are impractical to implement in Python alone, such as launching distributed training jobs on GPU clusters via scripts/train/newtrainer-frontier.sh and executing standardized benchmark suites through scripts/run_benchmark.sh.

Does olmocr provide Docker support?

Yes. The repository includes a Dockerfile that leverages Python 3.11-slim as its base image, installing system dependencies and the Python package via pip. This enables containerized execution of the OCR pipeline across different operating systems and computing environments.

Are there any other programming languages like C++ or JavaScript in the codebase?

No. The allenai/olmocr codebase does not contain C++, JavaScript, or other compiled programming languages. While olmocr/viewer/dolmaviewer_template.html contains HTML markup, it functions as a template utilized by Python viewer code rather than a standalone JavaScript application.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}