How to Specify a Different Model for olmOCR: 3 Methods Explained

You can specify a different model for olmOCR by using the --model CLI flag, passing a custom identifier when connecting to remote servers, or setting args.model programmatically when importing the Python pipeline module.

The olmOCR inference pipeline in the allenai/olmocr repository uses a configurable model identifier to determine which checkpoint performs the PDF-to-markdown conversion. This identifier is injected into every request sent to the underlying VLLM or OpenAI-compatible server, allowing you to swap default weights for custom fine-tunes or alternative model architectures without modifying core logic.

Method 1: Pass a Custom Model via the CLI --model Flag

The most direct way to specify a different model for olmOCR is through the command-line interface. In olmocr/pipeline.py (lines 12–15), the argument parser defines a --model option that defaults to allenai/olmOCR-2-7B-1025-FP8 but accepts any Hugging Face model ID or local path recognized by your inference backend.

To override the default, provide your custom model identifier when invoking the pipeline:

olmocr ./my_workspace \
    --model myorg/olmOCR-7B-custom \
    --pdfs docs/*.pdf \
    --markdown

This string is stored in args.model and propagated to the request builder, which embeds it in the JSON payload sent to the inference server.

Method 2: Configure Remote Inference Endpoints

When pointing olmOCR at an external OpenAI-compatible endpoint using the --server flag, the model identifier you supply via --model is forwarded directly to that remote host. In olmocr/pipeline.py (lines 106–135), the build_page_query function constructs the request payload and explicitly sets the "model" field to the value of args.model.

The remote server must recognize this identifier—whether it is a Hugging Face repo ID or a provider-specific alias:

olmocr ./my_workspace \
    --server http://my-vllm-host:8000/v1 \
    --model myorg/olmOCR-7B-custom \
    --markdown \
    --pdfs s3://my-bucket/pdfs/*.pdf

Because the pipeline treats the model name as an opaque string in remote mode, you can target specialized endpoints hosting quantized or fine-tuned variants without local GPU access.

Method 3: Set the Model Programmatically in Python

For custom workflows that import the pipeline as a library, you can override the model by manipulating the argument namespace before invoking main() or process_page(). The args.model attribute is referenced throughout olmocr/pipeline.py, including in build_page_query (lines 106–115), where it is inserted into the request body.

Here is how to invoke the pipeline programmatically with a non-default model:

from olmocr.pipeline import main
import sys
import asyncio

# Simulate CLI arguments with a custom model

sys.argv = [
    "pipeline.py",
    "./my_workspace",
    "--model", "myorg/olmOCR-7B-custom",
    "--pdfs", "sample.pdf",
]

# Run the pipeline

asyncio.run(main())

This approach allows dynamic model selection based on runtime conditions or configuration files.

How the Model Identifier Works Under the Hood

The model string you provide is used exclusively when constructing the request payload for the VLLM server. When olmOCR launches its local server via vllm_server_task in olmocr/pipeline.py (lines 808–819), it specifies --served-model-name olmocr as a generic alias, but the actual checkpoint loaded is determined by the download_model function using your provided identifier.

Changing the model identifier does not require code changes or reinstallation. The pipeline simply passes the string through to the underlying inference engine, which handles checkpoint resolution, downloading, and loading.

Summary

• CLI Override: Use --model followed by a Hugging Face ID or local path to replace the default allenai/olmOCR-2-7B-1025-FP8 checkpoint.
• Remote Inference: Combine --server and --model to target external VLLM instances hosting custom models.
• Programmatic Control: Set args.model on the argument namespace before calling main() or process_page() when using the Python API.
• Internal Flow: The identifier is inserted into the JSON payload in build_page_query and consumed by the VLLM server regardless of where the model is hosted.

Frequently Asked Questions

Can I use a local fine-tuned checkpoint instead of a Hugging Face model ID?

Yes. The --model argument accepts both Hugging Face repository identifiers (e.g., myorg/olmOCR-7B-custom) and absolute local paths to model directories. The download_model function in olmocr/pipeline.py handles both cases, loading weights from disk if the path exists locally or fetching from the Hugging Face Hub if remote.

Does changing the model require restarting a running VLLM server?

If you are using olmOCR's built-in local inference mode, the pipeline automatically manages the server lifecycle and will launch a new VLLM instance with the specified model. If you are connecting to an external server via --server, you must ensure that target server already hosts the model you specify, or restart it to load the new checkpoint.

What model format does olmOCR expect?

olmOCR expects models compatible with the VLLM serving engine, typically Llama-based architectures or other transformer variants supported by VLLM's --model parameter. The pipeline itself is agnostic to the specific architecture as long as the inference server can load the checkpoint and expose it via an OpenAI-compatible chat completions API.

Is the --model flag available in other olmOCR scripts besides the main pipeline?

Yes. The convention is consistent across the repository. For example, scripts/pii/tagging_pipeline.py (line 677) also exposes a --model argument following the same pattern, indicating that auxiliary tools in the allenai/olmocr ecosystem support identical model override behavior.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}