# How to Evaluate PaddleOCR Model Performance: A Complete Guide

> Master PaddleOCR model performance evaluation. Learn to use tools eval.py or program.eval() for accurate recognition and detection metrics. Boost your OCR project's success.

- Repository: [PaddlePaddle/PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR)
- Tags: how-to-guide
- Published: 2026-03-03

---

**Use [`tools/eval.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/eval.py) to run standardized evaluation on validation data, or call `program.eval()` directly in Python to compute task-specific metrics like accuracy for recognition or H-mean for detection.**

PaddleOCR provides a unified evaluation framework that handles both single-task models (text detection, recognition, classification) and complex end-to-end document understanding pipelines. The evaluation system is configuration-driven, allowing you to assess model quality using YAML config files without modifying source code, while the underlying implementation in [`tools/program.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/program.py) provides a reusable loop for custom integrations.

## Overview of the Evaluation Architecture

The PaddleOCR evaluation flow follows a strict pipeline: data loading → inference → post-processing → metric aggregation. This architecture is implemented across several core modules:

- **[`tools/eval.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/eval.py)** – The command-line entry point that orchestrates the entire process (lines 35-78).
- **[`tools/program.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/program.py)** – Contains the generic `eval()` function (lines 61-74) that executes the inference loop.
- **`ppocr/metrics/`** – Task-specific metric classes (`RecMetric`, `DetMetric`, `E2EMetric`) that compute final scores.
- **[`tools/end2end/eval_end2end.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/end2end/eval_end2end.py)** – Specialized evaluator for full pipelines that calculates polygon IoU and edit distance.

For single-task evaluation, the system uses `build_dataloader()` to prepare validation data, `build_model()` to construct the architecture, and `build_metric()` to instantiate the appropriate scorer based on the config file's `Metric` section.

## Single-Task Model Evaluation

Single-task evaluation applies to text detection, text recognition, and angle classification models. You can run evaluation via CLI for quick assessment or use the Python API for custom workflows.

### Command-Line Evaluation

The fastest way to evaluate a trained model is using the provided evaluation script with a configuration file:

```bash

# Install PaddleOCR

pip install "paddleocr[all]"

# Run evaluation (example for recognition model)

python tools/eval.py -c configs/rec/rec_ch_PP-OCRv3.yml

```

Ensure your configuration file contains a valid `Eval` dataset section pointing to your validation images and annotations. The script automatically loads the checkpoint specified in the config, runs inference on the validation set, and outputs metrics:

```

metric eval ***************
accuracy:0.9578
norm_edit_dis:0.0412
fps: 123.4

```

The [`tools/eval.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/eval.py) script handles device placement, mixed precision (AMP) scaling when enabled, and checkpoint loading via `load_model()` from [`ppocr/utils/save_load.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/ppocr/utils/save_load.py).

### Python API Evaluation

For integration into custom pipelines or Jupyter notebooks, import the evaluation components directly:

```python
import yaml
import paddle
from ppocr.data import build_dataloader
from ppocr.modeling.architectures import build_model
from ppocr.postprocess import build_post_process
from ppocr.metrics import build_metric
from ppocr.utils.save_load import load_model
from tools.program import eval as paddleocr_eval

# Load configuration

config = yaml.safe_load(open('configs/rec/rec_ch_PP-OCRv3.yml', 'rb'))
device = paddle.set_device('gpu' if paddle.is_compiled_with_cuda() else 'cpu')

# Build components

val_loader = build_dataloader(config, 'Eval', device, logger=None)
model = build_model(config['Architecture'])
load_model(config, model, model_type=config['Architecture']['model_type'])
post_process = build_post_process(config['PostProcess'], config['Global'])
metric = build_metric(config['Metric'])

# Execute evaluation

results = paddleocr_eval(
    model,
    val_loader,
    post_process,
    metric,
    model_type=config['Architecture'].get('model_type')
)
print(f"Final accuracy: {results}")

```

This approach gives you full control over the data loader, model initialization, and metric computation while maintaining compatibility with PaddleOCR's standardized evaluation logic.

### Key Evaluation Components

Understanding these factory functions helps debug evaluation issues or implement custom metrics:

- **`build_dataloader(config, 'Eval', device, ...)`** – Constructs the validation data loader from [`ppocr/data/__init__.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/ppocr/data/__init__.py) using the dataset paths and transforms defined in the config's `Eval` section.
- **`build_model(config['Architecture'])`** – Instantiates the neural network architecture; for distillation models, this automatically adjusts head output channels.
- **`build_post_process(config['PostProcess'], global_config)`** – Converts raw model outputs (logits or heatmaps) into interpretable predictions like text strings or bounding boxes.
- **`build_metric(config['Metric'])`** – Creates the metric calculator that aggregates batch-wise statistics into final scores such as **accuracy** for recognition or **H-mean** for detection.

## End-to-End Document Evaluation

For complete OCR pipelines that include layout analysis, text detection, and recognition in one system, use the specialized end-to-end evaluator:

```bash
python tools/end2end/eval_end2end.py \
    /path/to/ground_truth \
    /path/to/predictions \
    --ignore_blank=True

```

This script computes **polygon IoU** (Intersection over Union) between predicted and ground-truth text regions using the `polygon_iou()` function, then calculates **character-level edit distance** via the `ed()` function. It aggregates these into precision, recall, and F-measure scores:

```

hit, dt_count, gt_count  1250 1300 1240
character_acc: 93.21%
avg_edit_dist_field: 1.84
precision: 96.15%
recall: 92.31%
fmeasure: 94.19%

```

The `e2e_eval()` function (lines 70-80 in [`tools/end2end/eval_end2end.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/end2end/eval_end2end.py)) handles the complex matching logic required when multiple text regions overlap or when layout elements must be considered alongside text content.

## Understanding Evaluation Metrics

Different tasks use different primary indicators:

- **Text Recognition**: Uses `RecMetric` to calculate **accuracy** (exact match) and **normalized edit distance** (character error rate).
- **Text Detection**: Uses `DetMetric` to compute **precision**, **recall**, and **H-mean** (F1-score) based on box IoU thresholds.
- **End-to-End**: Uses `E2EMetric` or the standalone [`eval_end2end.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/eval_end2end.py) to combine detection IoU with transcription accuracy, producing an overall **F-measure** that reflects both localization and recognition quality.

All metric classes implement a consistent interface: they accumulate batch results via `__call__()` and return a dictionary of final values through `get_metric()`.

## Summary

- **Primary entry point**: Use [`tools/eval.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/eval.py) for standard single-task evaluation and [`tools/end2end/eval_end2end.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/end2end/eval_end2end.py) for full document pipelines.
- **Configuration-driven**: Evaluation requires a YAML config file with a properly defined `Eval` dataset section and `Metric` specification.
- **Core workflow**: `build_dataloader()` → `build_model()` → `load_model()` → `program.eval()` → metric aggregation.
- **Key files**: [`tools/program.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/program.py) contains the reusable evaluation loop; `ppocr/metrics/` contains task-specific scoring logic.
- **Metrics vary by task**: Recognition uses accuracy, detection uses H-mean, and end-to-end uses F-measure combined with edit distance.

## Frequently Asked Questions

### How do I evaluate a model on a custom dataset?

Prepare a YAML configuration file that specifies your validation image paths and label files in the `Eval.dataset` section, then run `python tools/eval.py -c your_config.yml`. The data loader will automatically use your custom annotations as long as they follow the format expected by the dataset class (e.g., `SimpleDataSet` or `LMDBDataSet`).

### What is the difference between single-task and end-to-end evaluation?

**Single-task evaluation** ([`tools/eval.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/eval.py)) assesses one component at a time—either detection, recognition, or classification—using task-specific metrics like accuracy or H-mean. **End-to-end evaluation** ([`tools/end2end/eval_end2end.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/end2end/eval_end2end.py)) assesses the complete pipeline by matching predicted text regions against ground truth using polygon IoU, then checking transcription accuracy, providing a holistic F-measure score.

### Can I use mixed precision (AMP) during evaluation?

Yes. When running `python tools/eval.py`, the evaluation loop in [`tools/program.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/program.py) automatically detects and uses Automatic Mixed Precision if configured in your YAML file's `Global` section. This speeds up inference on supported GPUs without affecting metric accuracy.

### How do I add a custom metric for a new task?

Create a new metric class in `ppocr/metrics/` (e.g., [`custom_metric.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/custom_metric.py)) that implements `__call__(self, preds, batch)` to process batch results and `get_metric()` to return final scores. Register it in [`ppocr/metrics/__init__.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/ppocr/metrics/__init__.py), then reference it in your config file's `Metric` section. The existing [`tools/eval.py`](https://github.com/PaddlePaddle/PaddleOCR/blob/main/tools/eval.py) driver will automatically use your custom metric without requiring changes to the evaluation script.