# How to Specify a Different Model for olmOCR: 3 Methods Explained

> Learn how to specify a different model for olmOCR with three easy methods. Use CLI flags, remote server identifiers, or programmatic control for flexible OCR.

- Repository: [Ai2/olmocr](https://github.com/allenai/olmocr)
- Tags: how-to-guide
- Published: 2026-07-02

---

**You can specify a different model for olmOCR by using the `--model` CLI flag, passing a custom identifier when connecting to remote servers, or setting `args.model` programmatically when importing the Python pipeline module.**

The olmOCR inference pipeline in the `allenai/olmocr` repository uses a configurable model identifier to determine which checkpoint performs the PDF-to-markdown conversion. This identifier is injected into every request sent to the underlying VLLM or OpenAI-compatible server, allowing you to swap default weights for custom fine-tunes or alternative model architectures without modifying core logic.

## Method 1: Pass a Custom Model via the CLI `--model` Flag

The most direct way to specify a different model for olmOCR is through the command-line interface. In [`olmocr/pipeline.py`](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py) (lines 12–15), the argument parser defines a `--model` option that defaults to `allenai/olmOCR-2-7B-1025-FP8` but accepts any Hugging Face model ID or local path recognized by your inference backend.

To override the default, provide your custom model identifier when invoking the pipeline:

```bash
olmocr ./my_workspace \
    --model myorg/olmOCR-7B-custom \
    --pdfs docs/*.pdf \
    --markdown

```

This string is stored in `args.model` and propagated to the request builder, which embeds it in the JSON payload sent to the inference server.

## Method 2: Configure Remote Inference Endpoints

When pointing olmOCR at an external OpenAI-compatible endpoint using the `--server` flag, the model identifier you supply via `--model` is forwarded directly to that remote host. In [`olmocr/pipeline.py`](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py) (lines 106–135), the `build_page_query` function constructs the request payload and explicitly sets the `"model"` field to the value of `args.model`.

The remote server must recognize this identifier—whether it is a Hugging Face repo ID or a provider-specific alias:

```bash
olmocr ./my_workspace \
    --server http://my-vllm-host:8000/v1 \
    --model myorg/olmOCR-7B-custom \
    --markdown \
    --pdfs s3://my-bucket/pdfs/*.pdf

```

Because the pipeline treats the model name as an opaque string in remote mode, you can target specialized endpoints hosting quantized or fine-tuned variants without local GPU access.

## Method 3: Set the Model Programmatically in Python

For custom workflows that import the pipeline as a library, you can override the model by manipulating the argument namespace before invoking `main()` or `process_page()`. The `args.model` attribute is referenced throughout [`olmocr/pipeline.py`](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py), including in `build_page_query` (lines 106–115), where it is inserted into the request body.

Here is how to invoke the pipeline programmatically with a non-default model:

```python
from olmocr.pipeline import main
import sys
import asyncio

# Simulate CLI arguments with a custom model

sys.argv = [
    "pipeline.py",
    "./my_workspace",
    "--model", "myorg/olmOCR-7B-custom",
    "--pdfs", "sample.pdf",
]

# Run the pipeline

asyncio.run(main())

```

This approach allows dynamic model selection based on runtime conditions or configuration files.

## How the Model Identifier Works Under the Hood

The model string you provide is used exclusively when constructing the request payload for the VLLM server. When olmOCR launches its local server via `vllm_server_task` in [`olmocr/pipeline.py`](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py) (lines 808–819), it specifies `--served-model-name olmocr` as a generic alias, but the actual checkpoint loaded is determined by the `download_model` function using your provided identifier.

Changing the model identifier does not require code changes or reinstallation. The pipeline simply passes the string through to the underlying inference engine, which handles checkpoint resolution, downloading, and loading.

## Summary

- **CLI Override**: Use `--model` followed by a Hugging Face ID or local path to replace the default `allenai/olmOCR-2-7B-1025-FP8` checkpoint.
- **Remote Inference**: Combine `--server` and `--model` to target external VLLM instances hosting custom models.
- **Programmatic Control**: Set `args.model` on the argument namespace before calling `main()` or `process_page()` when using the Python API.
- **Internal Flow**: The identifier is inserted into the JSON payload in `build_page_query` and consumed by the VLLM server regardless of where the model is hosted.

## Frequently Asked Questions

### Can I use a local fine-tuned checkpoint instead of a Hugging Face model ID?

Yes. The `--model` argument accepts both Hugging Face repository identifiers (e.g., `myorg/olmOCR-7B-custom`) and absolute local paths to model directories. The `download_model` function in [`olmocr/pipeline.py`](https://github.com/allenai/olmocr/blob/main/olmocr/pipeline.py) handles both cases, loading weights from disk if the path exists locally or fetching from the Hugging Face Hub if remote.

### Does changing the model require restarting a running VLLM server?

If you are using olmOCR's built-in local inference mode, the pipeline automatically manages the server lifecycle and will launch a new VLLM instance with the specified model. If you are connecting to an external server via `--server`, you must ensure that target server already hosts the model you specify, or restart it to load the new checkpoint.

### What model format does olmOCR expect?

olmOCR expects models compatible with the VLLM serving engine, typically Llama-based architectures or other transformer variants supported by VLLM's `--model` parameter. The pipeline itself is agnostic to the specific architecture as long as the inference server can load the checkpoint and expose it via an OpenAI-compatible chat completions API.

### Is the `--model` flag available in other olmOCR scripts besides the main pipeline?

Yes. The convention is consistent across the repository. For example, [`scripts/pii/tagging_pipeline.py`](https://github.com/allenai/olmocr/blob/main/scripts/pii/tagging_pipeline.py) (line 677) also exposes a `--model` argument following the same pattern, indicating that auxiliary tools in the `allenai/olmocr` ecosystem support identical model override behavior.