# How to Choose Between MinerU's Pipeline, VLM, and Hybrid Backends: A Complete Guide

> Choose the right MinerU backend for your needs. Understand when to use pipeline, VLM, or hybrid for CPU, low-VRAM, or high-end GPU environments with VLM accuracy.

- Repository: [OpenDataLab/MinerU](https://github.com/opendatalab/mineru)
- Tags: how-to-guide
- Published: 2026-02-22

---

**Use the pipeline backend for CPU-only or low-VRAM environments (6GB+), the VLM backend for maximum accuracy on high-end GPUs (8GB+ VRAM), and the hybrid backend to combine VLM-grade layout detection with proven OCR and formula extraction while allowing remote server offload.**

MinerU supports three distinct parsing architectures—**pipeline**, **VLM**, and **hybrid**—each optimized for different hardware constraints and accuracy requirements. Choosing the right backend determines whether your document extraction runs on a laptop CPU, a single high-end GPU, or distributed across edge clients and central servers. This guide explains how to choose between MinerU's pipeline, VLM, and hybrid backends based on the `opendatalab/MinerU` source code and practical deployment scenarios.

## Architecture Overview

MinerU's three backends represent fundamentally different approaches to document parsing, ranging from classical computer vision cascades to modern vision-language models.

### Pipeline Backend

The **pipeline** backend implements a classic multi-model cascade: layout detection → OCR → formula recognition → table extraction. In [`mineru/backend/pipeline/pipeline_analyze.py`](https://github.com/opendatalab/MinerU/blob/main/mineru/backend/pipeline/pipeline_analyze.py), each stage uses a specialized lightweight model.

- **No hallucinations**: Layout derives from dedicated detection models rather than generative outputs
- **Full feature support**: Complete access to table merging, formula detection, and multilingual OCR post-processing
- **Minimal VRAM**: Runs on CPU-only machines or GPUs with as little as 6GB VRAM

### VLM Backend

The **VLM** (Vision-Language Model) backend processes documents in a single forward pass using a large vision-language model. The implementation in [`mineru/backend/vlm/vlm_analyze.py`](https://github.com/opendatalab/MinerU/blob/main/mineru/backend/vlm/vlm_analyze.py) handles layout analysis, OCR, formula extraction, and table recognition simultaneously.

- **Highest accuracy**: Superior performance on complex multi-column layouts and dense scientific papers
- **Single-pass inference**: Eliminates cascade overhead, reducing latency for individual documents
- **Hardware requirement**: Requires a GPU with **≥8GB VRAM** using vLLM, LMDeploy, or MLX engines

### Hybrid Backend

The **hybrid** backend combines VLM layout detection with the pipeline's specialized OCR, formula, and table modules. Orchestrated in [`mineru/backend/hybrid/hybrid_analyze.py`](https://github.com/opendatalab/MinerU/blob/main/mineru/backend/hybrid/hybrid_analyze.py), this approach runs the VLM to identify layout blocks, then crops regions for processing by the pipeline's proven sub-modules.

- **Balanced precision**: VLM-grade layout quality with pipeline stability for text recognition
- **Flexible deployment**: Supports local GPU execution (`hybrid-auto-engine`) or remote VLM servers (`hybrid-http-client`)
- **VRAM efficiency**: Runs on **≥6GB GPUs** locally, or shifts VLM load to remote servers while keeping clients lightweight

## How Backend Selection Works Internally

When you specify a backend via CLI or API, MinerU follows a strict decision flow implemented across several core files:

1. **Backend name parsing**: The CLI extracts the backend string (e.g., `-b hybrid-auto-engine`)

2. **Engine automatic selection**: For backends ending with `-auto-engine`, the `get_vlm_engine()` function in [`mineru/utils/engine_utils.py`](https://github.com/opendatalab/MinerU/blob/main/mineru/utils/engine_utils.py) selects the optimal inference engine:
   - vLLM on Linux
   - LMDeploy on Windows  
   - MLX on macOS
   - Falls back to Transformers if specialized engines are unavailable

3. **Hybrid execution path**: If the backend starts with `hybrid`, MinerU first runs the VLM model to obtain layout blocks, then feeds cropped image regions to the pipeline's OCR, formula, and table models

4. **VLM-only path**: For `vlm-auto-engine` or `vlm-http-client`, the entire document processes through the VLM model in [`mineru/backend/vlm/vlm_analyze.py`](https://github.com/opendatalab/MinerU/blob/main/mineru/backend/vlm/vlm_analyze.py)

5. **Pipeline path**: When using `pipeline`, the classic cascade executes via [`mineru/backend/pipeline/pipeline_analyze.py`](https://github.com/opendatalab/MinerU/blob/main/mineru/backend/pipeline/pipeline_analyze.py)

The system respects several environment variables during selection:

- `MINERU_HYBRID_BATCH_RATIO`: Controls batch size for OCR/formula models in hybrid mode to fit various GPU sizes
- `MINERU_FORCE_VLM_OCR_ENABLE`: Forces VLM-based OCR even on low-VRAM devices
- `MINERU_HYBRID_FORCE_PIPELINE_ENABLE`: Disables VLM OCR in hybrid mode for debugging purposes

## Decision Matrix: When to Use Each Backend

Match your deployment scenario to the appropriate backend using these specific criteria:

| Situation | Recommended Backend | Rationale |
|-----------|--------------------|-----------|
| **CPU-only** or **GPU <6GB VRAM** | `pipeline` | No VLM engine can be loaded; lightweight models fit the memory budget |
| **Maximum layout accuracy** required and **GPU ≥8GB** available | `vlm-auto-engine` or `vlm-http-client` | VLM yields superior layout detection on complex documents |
| **VLM layout quality + proven OCR/formula** needed with **GPU ≥6GB** | `hybrid-auto-engine` | Combines VLM layout precision with pipeline stability |
| **Edge devices** processing via remote service | `hybrid-http-client` | Client stays lightweight while delegating VLM work to a central server |
| **Fastest end-to-end speed** on single GPU | `vlm-auto-engine` | Single forward pass eliminates cascade overhead |
| **Deterministic, hallucination-free** results | `pipeline` | Layout model trained specifically for document structure without generative errors |

## Practical Implementation Examples

### CLI Backend Selection

Select your backend using the `-b` or `--backend` flag:

```bash

# Pure CPU pipeline (default for low-resource machines)

mineru -p mydoc.pdf -o out_dir -b pipeline

# VLM backend with automatic engine selection (requires 8GB+ GPU)

mineru -p mydoc.pdf -o out_dir -b vlm-auto-engine

# Hybrid local engine (6GB+ GPU)

mineru -p mydoc.pdf -o out_dir -b hybrid-auto-engine

# Hybrid remote client (edge device talking to central server)

mineru -p mydoc.pdf -o out_dir -b hybrid-http-client -u http://192.168.1.10:30000

```

### Python API Integration

Explicitly set the backend when initializing the MinerU class:

```python
from mineru import MinerU

# CPU-friendly pipeline

miner = MinerU(backend="pipeline")

# High-accuracy VLM (requires GPU)

# miner = MinerU(backend="vlm-auto-engine")

# Balanced hybrid approach

# miner = MinerU(backend="hybrid-auto-engine")

# Parse document

mid_json, results, vlm_ocr = miner.parse(pdf_path="mydoc.pdf")

```

### Environment Variable Configuration

Configure defaults without CLI flags:

```bash
export MINERU_BACKEND=hybrid-auto-engine        # Default backend for all commands

export MINERU_HYBRID_BATCH_RATIO=4             # Reduce VRAM usage on 6GB cards

export MINERU_FORCE_VLM_OCR_ENABLE=true        # Force VLM OCR even on low-VRAM

mineru -p mydoc.pdf -o out_dir                  # Uses environment settings

```

### Remote VLM Server Deployment

For distributed deployments, run a central VLM server and lightweight clients:

```bash

# On GPU-rich server node

mineru-openai-server --engine vllm --model MinerU2.5 --host 0.0.0.0 --port 30000

# On edge client machine

mineru -p mydoc.pdf -o out_dir -b hybrid-http-client -u http://<server_ip>:30000

```

## Summary

- **Pipeline backend**: Choose for CPU-only environments, deterministic output, and minimal VRAM requirements (6GB+ if using GPU acceleration)
- **VLM backend**: Select when you have 8GB+ VRAM and require maximum layout accuracy on complex documents, accepting higher resource consumption for single-pass inference
- **Hybrid backend**: Use to balance VLM-grade layout detection with proven OCR and formula extraction, enabling both local 6GB GPU execution and remote server offload via HTTP client mode
- **Engine selection**: The `get_vlm_engine()` function in [`mineru/utils/engine_utils.py`](https://github.com/opendatalab/MinerU/blob/main/mineru/utils/engine_utils.py) automatically selects vLLM, LMDeploy, or MLX based on your operating system when using `-auto-engine` variants

## Frequently Asked Questions

### What is the main difference between MinerU's pipeline and VLM backends?

The **pipeline** backend uses a cascade of specialized models (layout → OCR → formula → table) executed sequentially in [`mineru/backend/pipeline/pipeline_analyze.py`](https://github.com/opendatalab/MinerU/blob/main/mineru/backend/pipeline/pipeline_analyze.py), providing deterministic results without generative hallucinations. The **VLM** backend processes the entire document through a single vision-language model in [`mineru/backend/vlm/vlm_analyze.py`](https://github.com/opendatalab/MinerU/blob/main/mineru/backend/vlm/vlm_analyze.py), offering higher accuracy on complex layouts but requiring 8GB+ VRAM and producing generative outputs.

### Can I run the MinerU VLM backend on a CPU-only machine?

No. The VLM backend requires a GPU with at least 8GB VRAM to load the vision-language model using vLLM, LMDeploy, or MLX engines. For CPU-only deployments, use the `pipeline` backend, or use `hybrid-http-client` mode to process documents on a CPU while offloading VLM inference to a remote GPU server.

### How does the hybrid backend reduce VRAM usage compared to pure VLM mode?

The **hybrid** backend in [`mineru/backend/hybrid/hybrid_analyze.py`](https://github.com/opendatalab/MinerU/blob/main/mineru/backend/hybrid/hybrid_analyze.py) uses the VLM only for layout detection, then processes cropped regions through the pipeline's lightweight OCR and formula models. This reduces VRAM requirements to 6GB for local execution. Additionally, `hybrid-http-client` mode allows multiple lightweight CPU clients to share a single VLM server, eliminating per-client GPU memory entirely.

### When should I use hybrid-http-client versus hybrid-auto-engine?

Use **hybrid-auto-engine** when running MinerU on a local GPU with 6GB+ VRAM, allowing automatic engine selection via [`mineru/utils/engine_utils.py`](https://github.com/opendatalab/MinerU/blob/main/mineru/utils/engine_utils.py). Choose **hybrid-http-client** when deploying to edge devices without GPUs or when consolidating GPU resources—this mode sends layout detection requests to a remote OpenAI-compatible server (e.g., `mineru-openai-server`) while running OCR and formula extraction locally.