How olmOCR Handles Mathematical Equations and LaTeX Rendering

OlmOCR treats mathematical equations as first-class content by preserving LaTeX delimiters throughout the OCR pipeline, rendering them with KaTeX in a headless browser, and validating output using MathML containment and geometric span-matching algorithms.

OlmOCR, developed by the Allen Institute for AI, converts PDF documents to structured HTML with specialized handling for mathematical notation. Unlike standard OCR pipelines that flatten equations into Unicode characters or raster images, olmOCR maintains the raw LaTeX source code and validates rendered output using a dedicated KaTeX-based rendering engine. This ensures that complex mathematical expressions survive the PDF-to-HTML conversion with perfect fidelity.

Detecting and Preserving LaTeX Delimiters

The HTML generation pipeline explicitly instructs the Claude model to wrap every formula in standard LaTeX delimiters. According to the source code in olmocr/synth/mine_html_templates.py (lines 95-96), the prompt requires inline equations to use \(...\) and display equations to use \[...\].

The generated HTML retains these raw LaTeX strings intact. While the pipeline later reconverts <sup> and <sub> tags to Unicode for regular text content, it leaves the LaTeX delimiters untouched (lines 94-97). This preservation strategy ensures that mathematical markup remains machine-readable and renderable throughout the downstream processing stages.

Rendering Equations with KaTeX

Before converting HTML back to PDF, olmOCR injects KaTeX's CSS and JavaScript assets into a Playwright-controlled headless browser. The _load_katex_on_page function (lines 84-101 in olmocr/synth/mine_html_templates.py) loads the KaTeX library and invokes renderMathInElement using the same delimiters defined during generation: \(...\) for inline math and \[...\] for display math.

This rendering approach produces high-fidelity visual representations while maintaining the underlying LaTeX source code for subsequent validation steps.

Validating Mathematical Output

The validation layer relies on a custom KaTeX-based test harness defined in olmocr/bench/katex/render.py. The render_equation function (lines 28-74) loads KaTeX assets in a headless Chromium page, renders a LaTeX string, and extracts both the generated MathML and the innermost <span> elements with their precise bounding boxes.

For comparison, compare_rendered_equations (lines 14-50) implements a two-stage validation algorithm:

1. MathML Normalization: Normalizes the MathML structure and checks for containment relationships between reference and hypothesis outputs.
2. Geometric Span Matching: If the MathML comparison fails, the system falls back to a neighbor-based algorithm that verifies token-level geometry—specifically the up, down, left, and right spatial relationships between spans.

Integration with the Test Suite

The test suite in tests/test_katex_render.py exercises the renderer across numerous edge cases, including inline versus display modes, color wrappers, \newcommand definitions, and matrix environments.

When building text-presence tests, the pipeline specifically strips LaTeX blocks from intermediate markdown using regex patterns that remove $$...$$, \[...\], and \(...\) sequences (lines 66-73 in mine_html_templates.py). This ensures that equations are evaluated exclusively by the specialized KaTeX-based matcher rather than plain-text validators, preventing false mismatches on complex mathematical notation.

Practical Implementation

The following examples demonstrate the core API for rendering and validating equations:

# Generate HTML from Claude - equations stay wrapped in \[ ... \] or \( ... \)

html = await generate_html_from_image(client, image_base64)

# Load KaTeX on the page before PDF rendering

await _load_katex_on_page(page)  # See lines 84-101 in mine_html_templates.py

from olmocr.bench.katex import render_equation, compare_rendered_equations

# Render reference and hypothesis equations

eq_ref = render_equation(r"\int_{a}^{b} f(x)\,dx")
eq_hyp = render_equation(r"\int\limits_{a}^{b} f(x)dx")

# Compare using MathML and geometric span matching

assert compare_rendered_equations(eq_ref, eq_hyp)  # Returns True

Key Implementation Files

Component	File	Purpose
HTML generation & LaTeX delimiters	olmocr/synth/mine_html_templates.py (lines 84-101)	Instructs Claude to preserve LaTeX and loads KaTeX before PDF rendering
KaTeX rendering engine	olmocr/bench/katex/render.py (lines 28-74)	Renders LaTeX and extracts MathML with span geometry
Equation comparison logic	olmocr/bench/katex/render.py (lines 14-50)	Normalizes MathML and performs span-based matching
Test suite	tests/test_katex_render.py	Validates rendering across edge cases
Text test integration	olmocr/synth/mine_html_templates.py (lines 66-73)	Strips LaTeX from text-only tests to isolate math validation

Summary

• OlmOCR preserves LaTeX source code by requiring explicit \(...\) and \[...\] delimiters in the HTML generation prompt.
• Mathematical equations render via KaTeX in a headless Playwright browser before final PDF conversion.
• Validation uses both MathML containment checks and geometric span-matching algorithms to verify equation accuracy.
• The test suite isolates mathematical validation from plain-text checks by stripping LaTeX blocks from text-presence tests.

Frequently Asked Questions

Does olmOCR convert mathematical equations to Unicode characters?

No. Unlike traditional OCR systems that convert superscripts and subscripts to Unicode characters, olmOCR preserves the raw LaTeX source code. The pipeline specifically avoids converting mathematical delimiters to HTML tags, ensuring that equations remain in their original LaTeX format for accurate rendering and validation.

Which LaTeX delimiters does olmOCR use for inline and display math?

OlmOCR uses \(...\) for inline mathematical expressions and \[...\] for display equations (block-level math). These delimiters are explicitly required in the Claude prompt at olmocr/synth/mine_html_templates.py (lines 95-96) and are used consistently throughout the rendering and validation pipeline.

How does olmOCR validate that rendered equations match the original PDF?

The validation uses a two-tier approach implemented in olmocr/bench/katex/render.py. First, it normalizes and compares the MathML structure. If that fails, it falls back to a geometric span-matching algorithm that verifies the spatial relationships (up, down, left, right) between individual tokens in the reference and rendered equations.

Why does olmOCR strip LaTeX from text-presence tests?

The pipeline removes LaTeX blocks (matching $$...$$, \[...\], and \(...\) patterns) from intermediate markdown at lines 66-73 of mine_html_templates.py to prevent plain-text validators from flagging mathematical notation as errors. This ensures that equations are evaluated only by the specialized KaTeX-based matcher, which understands mathematical structure and rendering.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}