# How to Use GPU with PaddleOCR: Complete Device Configuration Guide

> Boost PaddleOCR performance with GPU acceleration. Follow this guide to configure your device for faster OCR processing using the paddlepaddle-gpu package and simple command line or Python setup.

- Repository: [PaddlePaddle/PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR)
- Tags: how-to-guide
- Published: 2026-03-03

---

**Configure GPU acceleration in PaddleOCR by passing `--device=gpu` to CLI commands or calling `paddle.set_device("gpu")` in Python scripts, after installing the `paddlepaddle-gpu` package.**

PaddleOCR delivers high-performance optical character recognition that benefits significantly from GPU acceleration. While the framework automatically detects CUDA-capable hardware at runtime, explicit control over device selection ensures optimal utilization across single-GPU, multi-GPU, and CPU-only environments.

## How Device Selection Works in PaddleOCR

PaddleOCR implements a **device abstraction layer** that translates high-level user preferences into PaddlePaddle runtime configuration. The selection flow follows four distinct stages defined in the source code:

- **CLI Parsing** – The [`paddleocr/_common_args.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/paddleocr/_common_args.py) file (lines 103-107) defines the `--device` argument with a default value of `"gpu"` when CUDA is available.
- **Device String Construction** – [`tools/program.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/program.py) (lines 119-124 and 923-926) converts the parsed flag into a concrete device identifier (`gpu:0`, `cpu`, `xpu:0`, etc.) based on the parallel environment.
- **Runtime Configuration** – The same file calls `paddle.set_device()` (lines 928-929) to bind all subsequent operations to the specified accelerator.
- **Inference Forwarding** – Entry points like [`tools/infer_rec.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/infer_rec.py) (lines 41-44) pass these arguments through to the common program handler, ensuring consistent behavior across detection, recognition, and classification tasks.

If the installed PaddlePaddle build lacks CUDA support, the framework raises a runtime error when `--device=gpu` is requested, with diagnostic messages implemented in [`tools/program.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/program.py) (lines 140-144).

## Installing GPU-Compatible PaddlePaddle

Before configuring device selection, install a CUDA-enabled PaddlePaddle build. PaddleOCR does not include GPU support directly; it depends on the underlying Paddle framework.

Install the GPU version matching your CUDA driver:

```bash

# For CUDA 11.x environments

python -m pip install "paddlepaddle-gpu>=2.5,<2.6" -i https://mirror.baidu.com/pypi/simple

# Install PaddleOCR itself

python -m pip install paddleocr

```

Verify CUDA compilation status before proceeding:

```python
import paddle
print(paddle.is_compiled_with_cuda())  # Must return True

```

## Command-Line GPU Configuration

PaddleOCR provides two methods for device specification via command line: the modern `--device` flag and the legacy `--use_gpu` option.

### Using the Modern `--device` Flag

The `--device` argument in [`paddleocr/_common_args.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/paddleocr/_common_args.py) supports explicit device identifiers:

- **`--device=gpu`** – Selects the first GPU (equivalent to `gpu:0`)
- **`--device=gpu:0,1`** – Enables multi-GPU parallel inference on devices 0 and 1
- **`--device=cpu`** – Forces CPU execution for debugging or compatibility

Example for single-GPU text recognition:

```bash
python tools/infer_rec.py \
    --image_dir ./doc_images \
    --device=gpu

```

### Legacy `--use_gpu` Flag

For backward compatibility, [`tools/infer/utility.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/infer/utility.py) (lines 41-48) maintains the `--use_gpu` boolean flag. When set to `True`, it selects `gpu:0` if available:

```bash
python tools/infer_det.py \
    --image_dir ./imgs \
    --use_gpu=True

```

## Multi-GPU Parallel Inference

For high-throughput scenarios, PaddleOCR supports distributing workloads across multiple GPUs using comma-separated device identifiers. When you specify `--device=gpu:0,1,2`, the framework spawns separate processes per GPU using Paddle's `fleet` distributed environment.

Each process receives its assigned device ID via `dist.ParallelEnv().dev_id` as implemented in [`tools/program.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/program.py).

Launch multi-GPU recognition with batch distribution:

```bash
python tools/infer_rec.py \
    --image_dir ./test_imgs \
    --device=gpu:0,1 \
    --batch_num 4

```

Each GPU processes its own batch of 4 images simultaneously. Ensure your input batch is sufficiently large to keep all GPUs utilized; small batches may leave secondary GPUs idle.

For distributed launching with `paddle.distributed.launch`:

```bash
python -m paddle.distributed.launch \
    --gpus "0,1,2" \
    tools/infer_rec.py \
    --image_dir ./large_dataset \
    --device=gpu

```

## Programmatic GPU Control in Python

For custom scripts using the PaddleOCR Python API, explicitly set the device before initializing the `PaddleOCR` class:

```python
import paddle
from paddleocr import PaddleOCR

# Force execution on GPU 1 (second physical device)

paddle.set_device("gpu:1")

ocr = PaddleOCR()  # Inherits the current device context

result = ocr.ocr("sample.jpg", cls=True)
print(result)

```

This approach bypasses CLI argument parsing and directly invokes the underlying `paddle.set_device()` mechanism referenced in [`tools/program.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/program.py) (lines 928-929).

## Troubleshooting Common GPU Issues

### CUDA Not Found Errors

**Symptom:** `paddle is not compiled with cuda` error when calling `paddle.set_device("gpu")`.

**Fix:** Install the GPU-enabled package (`paddlepaddle-gpu`) matching your driver version. Verify with `paddle.is_compiled_with_cuda()`.

### Out-of-Memory Crashes

**Symptom:** Process crashes during inference on large images.

**Fix:** Limit per-GPU memory allocation using the `--gpu_mem` parameter (e.g., `--gpu_mem=2000` for 2GB) or reduce `--batch_num`.

### Silent CPU Fallback

**Symptom:** GPU utilization remains at 0% despite requesting GPU execution.

**Fix:** Check that the Paddle binary reports CUDA support. If using the Python API, ensure `paddle.set_device()` is called before instantiating `PaddleOCR`.

## Summary

- **Install `paddlepaddle-gpu`** before attempting GPU acceleration; CPU-only builds raise runtime errors.
- **Use `--device=gpu`** for modern CLI workflows, or `--device=gpu:0,1` for multi-GPU inference.
- **Call `paddle.set_device()`** in Python scripts before instantiating the OCR engine to control device selection programmatically.
- **Reference implementation details** in [`paddleocr/_common_args.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/paddleocr/_common_args.py) (CLI options) and [`tools/program.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/program.py) (device initialization) when debugging configuration issues.

## Frequently Asked Questions

### How do I verify that PaddleOCR is using the GPU?

Run `python -c "import paddle; print(paddle.is_compiled_with_cuda())"` to confirm your Paddle installation supports CUDA. During inference, monitor GPU utilization with `nvidia-smi` or set the environment variable `CUDA_VISIBLE_DEVICES=0` to isolate the device.

### Can I use multiple GPUs to speed up single-image inference?

No. PaddleOCR's multi-GPU support distributes **batches** across devices, not single images. Each GPU processes distinct images simultaneously. For single-image latency reduction, use tensorRT optimization or reduce model precision rather than multi-GPU deployment.

### What is the difference between `--device` and `--use_gpu`?

The `--device` flag (defined in [`paddleocr/_common_args.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/paddleocr/_common_args.py)) accepts specific device strings like `gpu:0`, `cpu`, or `xpu`, while `--use_gpu` (in [`tools/infer/utility.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/infer/utility.py)) is a legacy boolean that only toggles `gpu:0` on or off. Modern workflows should prefer `--device` for explicit control.

### How do I force CPU execution when a GPU is present?

Pass `--device=cpu` to any inference script, or call `paddle.set_device("cpu")` before initializing `PaddleOCR`. This overrides the automatic GPU selection default and prevents CUDA initialization entirely.