How to Use GPU with PaddleOCR: Complete Device Configuration Guide

Configure GPU acceleration in PaddleOCR by passing --device=gpu to CLI commands or calling paddle.set_device("gpu") in Python scripts, after installing the paddlepaddle-gpu package.

PaddleOCR delivers high-performance optical character recognition that benefits significantly from GPU acceleration. While the framework automatically detects CUDA-capable hardware at runtime, explicit control over device selection ensures optimal utilization across single-GPU, multi-GPU, and CPU-only environments.

How Device Selection Works in PaddleOCR

PaddleOCR implements a device abstraction layer that translates high-level user preferences into PaddlePaddle runtime configuration. The selection flow follows four distinct stages defined in the source code:

• CLI Parsing – The paddleocr/_common_args.py file (lines 103-107) defines the --device argument with a default value of "gpu" when CUDA is available.
• Device String Construction – tools/program.py (lines 119-124 and 923-926) converts the parsed flag into a concrete device identifier (gpu:0, cpu, xpu:0, etc.) based on the parallel environment.
• Runtime Configuration – The same file calls paddle.set_device() (lines 928-929) to bind all subsequent operations to the specified accelerator.
• Inference Forwarding – Entry points like tools/infer_rec.py (lines 41-44) pass these arguments through to the common program handler, ensuring consistent behavior across detection, recognition, and classification tasks.

If the installed PaddlePaddle build lacks CUDA support, the framework raises a runtime error when --device=gpu is requested, with diagnostic messages implemented in tools/program.py (lines 140-144).

Installing GPU-Compatible PaddlePaddle

Before configuring device selection, install a CUDA-enabled PaddlePaddle build. PaddleOCR does not include GPU support directly; it depends on the underlying Paddle framework.

Install the GPU version matching your CUDA driver:

# For CUDA 11.x environments

python -m pip install "paddlepaddle-gpu>=2.5,<2.6" -i https://mirror.baidu.com/pypi/simple

# Install PaddleOCR itself

python -m pip install paddleocr

Verify CUDA compilation status before proceeding:

import paddle
print(paddle.is_compiled_with_cuda())  # Must return True

Command-Line GPU Configuration

PaddleOCR provides two methods for device specification via command line: the modern --device flag and the legacy --use_gpu option.

Using the Modern --device Flag

The --device argument in paddleocr/_common_args.py supports explicit device identifiers:

• --device=gpu – Selects the first GPU (equivalent to gpu:0)
• --device=gpu:0,1 – Enables multi-GPU parallel inference on devices 0 and 1
• --device=cpu – Forces CPU execution for debugging or compatibility

Example for single-GPU text recognition:

python tools/infer_rec.py \
    --image_dir ./doc_images \
    --device=gpu

Legacy --use_gpu Flag

For backward compatibility, tools/infer/utility.py (lines 41-48) maintains the --use_gpu boolean flag. When set to True, it selects gpu:0 if available:

python tools/infer_det.py \
    --image_dir ./imgs \
    --use_gpu=True

Multi-GPU Parallel Inference

For high-throughput scenarios, PaddleOCR supports distributing workloads across multiple GPUs using comma-separated device identifiers. When you specify --device=gpu:0,1,2, the framework spawns separate processes per GPU using Paddle's fleet distributed environment.

Each process receives its assigned device ID via dist.ParallelEnv().dev_id as implemented in tools/program.py.

Launch multi-GPU recognition with batch distribution:

python tools/infer_rec.py \
    --image_dir ./test_imgs \
    --device=gpu:0,1 \
    --batch_num 4

Each GPU processes its own batch of 4 images simultaneously. Ensure your input batch is sufficiently large to keep all GPUs utilized; small batches may leave secondary GPUs idle.

For distributed launching with paddle.distributed.launch:

python -m paddle.distributed.launch \
    --gpus "0,1,2" \
    tools/infer_rec.py \
    --image_dir ./large_dataset \
    --device=gpu

Programmatic GPU Control in Python

For custom scripts using the PaddleOCR Python API, explicitly set the device before initializing the PaddleOCR class:

import paddle
from paddleocr import PaddleOCR

# Force execution on GPU 1 (second physical device)

paddle.set_device("gpu:1")

ocr = PaddleOCR()  # Inherits the current device context

result = ocr.ocr("sample.jpg", cls=True)
print(result)

This approach bypasses CLI argument parsing and directly invokes the underlying paddle.set_device() mechanism referenced in tools/program.py (lines 928-929).

Troubleshooting Common GPU Issues

CUDA Not Found Errors

Symptom: paddle is not compiled with cuda error when calling paddle.set_device("gpu").

Fix: Install the GPU-enabled package (paddlepaddle-gpu) matching your driver version. Verify with paddle.is_compiled_with_cuda().

Out-of-Memory Crashes

Symptom: Process crashes during inference on large images.

Fix: Limit per-GPU memory allocation using the --gpu_mem parameter (e.g., --gpu_mem=2000 for 2GB) or reduce --batch_num.

Silent CPU Fallback

Symptom: GPU utilization remains at 0% despite requesting GPU execution.

Fix: Check that the Paddle binary reports CUDA support. If using the Python API, ensure paddle.set_device() is called before instantiating PaddleOCR.

Summary

• Install paddlepaddle-gpu before attempting GPU acceleration; CPU-only builds raise runtime errors.
• Use --device=gpu for modern CLI workflows, or --device=gpu:0,1 for multi-GPU inference.
• Call paddle.set_device() in Python scripts before instantiating the OCR engine to control device selection programmatically.
• Reference implementation details in paddleocr/_common_args.py (CLI options) and tools/program.py (device initialization) when debugging configuration issues.

Frequently Asked Questions

How do I verify that PaddleOCR is using the GPU?

Run python -c "import paddle; print(paddle.is_compiled_with_cuda())" to confirm your Paddle installation supports CUDA. During inference, monitor GPU utilization with nvidia-smi or set the environment variable CUDA_VISIBLE_DEVICES=0 to isolate the device.

Can I use multiple GPUs to speed up single-image inference?

No. PaddleOCR's multi-GPU support distributes batches across devices, not single images. Each GPU processes distinct images simultaneously. For single-image latency reduction, use tensorRT optimization or reduce model precision rather than multi-GPU deployment.

What is the difference between --device and --use_gpu?

The --device flag (defined in paddleocr/_common_args.py) accepts specific device strings like gpu:0, cpu, or xpu, while --use_gpu (in tools/infer/utility.py) is a legacy boolean that only toggles gpu:0 on or off. Modern workflows should prefer --device for explicit control.

How do I force CPU execution when a GPU is present?

Pass --device=cpu to any inference script, or call paddle.set_device("cpu") before initializing PaddleOCR. This overrides the automatic GPU selection default and prevents CUDA initialization entirely.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}