# How to Export an RF-DETR Model to TensorRT Format

> Easily export your RF-DETR model to TensorRT format. Learn how to convert trained detection models into optimized TensorRT engines using Roboflow's built-in pipeline.

- Repository: [Roboflow/rf-detr](https://github.com/roboflow/rf-detr)
- Tags: how-to-guide
- Published: 2026-09-08

---

**RF-DETR provides a built-in export pipeline that converts trained detection models into optimized TensorRT engines via an intermediate ONNX representation, accessible through both Python API and CLI interfaces.**

Exporting RF-DETR to TensorRT enables high-performance inference on NVIDIA GPUs. According to the `roboflow/rf-detr` source code, the export functionality is implemented in the core `RFDETR` class and orchestrated through dedicated export modules, eliminating the need for manual conversion scripts.

## Installing TensorRT Dependencies

TensorRT support is optional to keep the base installation lightweight. Install the required dependencies using the `tensorrt` extra, which pulls in the `polygraphy` wrapper library:

```bash
pip install "rfdetr[tensorrt]"

```

This extra ensures the **TensorRT Python API** is available through `polygraphy`, allowing in-process engine building without launching external `trtexec` subprocesses.

## Exporting via Python API

The `RFDETR.export()` method in [`src/rfdetr/detr.py`](https://github.com/roboflow/rf-detr/blob/main/src/rfdetr/detr.py) drives the entire conversion pipeline. This method handles ONNX generation and TensorRT engine serialization in a single call.

### Loading the Model

Instantiate a pre-trained model or load a custom checkpoint before export:

```python
import rfdetr

# Load a pre-trained small variant

model = rfdetr.detr.RFDETRSmall()

# Optional: configure input resolution (must be set before export)

model.resolution = (640, 640)
model.eval()  # Ensure deterministic export mode

```

### Configuring Export Parameters

The `export()` method accepts specific parameters for TensorRT optimization:

- **`format`**: Must be set to `"tensorrt"` to trigger the TensorRT builder
- **`fp16`**: Boolean flag for **FP16 precision** (default `True`). Set to `False` for FP32 engines when targeting TensorRT builds without FP16 support
- **`output_dir`**: Directory path for saving the ONNX intermediate and final `.engine` file
- **`engine_name`**: Base filename for the serialized engine (e.g., `"rfdetr_small"` generates `rfdetr_small-fp16.engine`)
- **`backbone_only`**: Export only the backbone architecture (useful for downstream fine-tuning pipelines)

### Executing the Export

Run the export with your desired configuration:

```python
engine_path = model.export(
    format="tensorrt",
    fp16=True,                # Set False for FP32 precision

    output_dir="./exported",  # Creates directory if missing

    engine_name="rfdetr_small"
)

print(f"TensorRT engine saved to: {engine_path}")

```

Under the hood, `RFDETR.export()` in [`src/rfdetr/detr.py`](https://github.com/roboflow/rf-detr/blob/main/src/rfdetr/detr.py) performs three operations:

1. **ONNX Generation** – Calls `torch.onnx.export` to create an ONNX graph, implemented in [`src/rfdetr/export/main.py`](https://github.com/roboflow/rf-detr/blob/main/src/rfdetr/export/main.py)
2. **Engine Building** – Passes the ONNX file to [`src/rfdetr/export/_tensorrt.py`](https://github.com/roboflow/rf-detr/blob/main/src/rfdetr/export/_tensorrt.py), which uses `polygraphy`'s `build_engine` API to create the serialized TensorRT engine
3. **Filename Resolution** – Uses utilities in [`src/rfdetr/export/_naming.py`](https://github.com/roboflow/rf-detr/blob/main/src/rfdetr/export/_naming.py) to generate consistent filenames (e.g., `<stem>-fp16.engine` or `<stem>-fp32.engine`)

## Exporting via Command Line

The same functionality is exposed through the CLI entry point in [`src/rfdetr/__main__.py`](https://github.com/roboflow/rf-detr/blob/main/src/rfdetr/__main__.py). This interface constructs the model via the factory pattern and forwards arguments to the Python API:

```bash
python -m rfdetr export \
    --model rfdetr-small \
    --format tensorrt \
    --fp16 \
    --output-dir ./exported \
    --engine-name rfdetr_small

```

Supported model names include `rfdetr-nano`, `rfdetr-small`, `rfdetr-medium`, and others. Omit the `--fp16` flag to generate an FP32 engine instead.

## Validating the TensorRT Engine

After export, verify engine functionality using the benchmark utility in [`src/rfdetr/export/benchmark.py`](https://github.com/roboflow/rf-detr/blob/main/src/rfdetr/export/benchmark.py):

```python
from rfdetr.export.benchmark import run_tensorrt_inference

results = run_tensorrt_inference(
    engine_path="exported/rfdetr_small-fp16.engine",
    image_path="sample.jpg"
)

```

This loads the engine using `polygraphy.trt.Engine.from_file`, executes forward passes, and reports latency statistics. The implementation mirrors the validation logic in [`tests/export/test_tensorrt_export.py`](https://github.com/roboflow/rf-detr/blob/main/tests/export/test_tensorrt_export.py), which ensures FP16 and FP32 export paths function correctly.

## Troubleshooting Common Issues

### FP16 Builder Limitations

Some minimal TensorRT installations lack FP16 support. The builder in [`src/rfdetr/export/_tensorrt.py`](https://github.com/roboflow/rf-detr/blob/main/src/rfdetr/export/_tensorrt.py) (lines 126-134) automatically detects this condition, falls back to FP32, and emits a warning rather than failing.

### ScatterND Compatibility

The transformer export path deliberately avoids `ScatterND` operations because TensorRT cannot consume them. As noted in [`src/rfdetr/models/transformer.py`](https://github.com/roboflow/rf-detr/blob/main/src/rfdetr/models/transformer.py) (lines 342-347), the implementation uses alternative graph constructions to ensure compatibility.

### GPU Availability Requirements

TensorRT export requires an NVIDIA GPU with CUDA drivers installed. The code checks for CUDA availability before invoking the builder and raises a clear error if running on CPU-only systems.

## Summary

- **Install** TensorRT support via `pip install "rfdetr[tensorrt]"`
- **Use** `model.export(format="tensorrt", ...)` from [`src/rfdetr/detr.py`](https://github.com/roboflow/rf-detr/blob/main/src/rfdetr/detr.py) for Python-based exports
- **Leverage** [`src/rfdetr/export/_tensorrt.py`](https://github.com/roboflow/rf-detr/blob/main/src/rfdetr/export/_tensorrt.py) for low-level engine building via `polygraphy`
- **Automate** exports using the CLI interface in [`src/rfdetr/__main__.py`](https://github.com/roboflow/rf-detr/blob/main/src/rfdetr/__main__.py)
- **Validate** engines with the benchmark utilities in [`src/rfdetr/export/benchmark.py`](https://github.com/roboflow/rf-detr/blob/main/src/rfdetr/export/benchmark.py)
- **Handle** FP16 limitations gracefully with automatic fallback to FP32 precision

## Frequently Asked Questions

### What is the difference between ONNX and TensorRT export in RF-DETR?

ONNX export produces a portable graph representation suitable for cross-platform deployment, while TensorRT export generates a highly optimized, NVIDIA-specific serialized engine. The TensorRT pipeline in [`src/rfdetr/export/main.py`](https://github.com/roboflow/rf-detr/blob/main/src/rfdetr/export/main.py) actually uses ONNX as an intermediate step, then immediately builds the TensorRT engine from that ONNX file using the builder in [`src/rfdetr/export/_tensorrt.py`](https://github.com/roboflow/rf-detr/blob/main/src/rfdetr/export/_tensorrt.py).

### Can I export RF-DETR to TensorRT on a machine without a GPU?

No. The TensorRT builder requires a physical NVIDIA GPU to optimize and serialize the engine, even if you later intend to run inference on a different device. The export code explicitly checks for CUDA availability and will raise a runtime error if no GPU is detected.

### How do I choose between FP16 and FP32 precision for TensorRT export?

Use **FP16** (the default) for maximum inference speed and reduced memory bandwidth, which is ideal for most modern NVIDIA GPUs (Turing architecture and newer). Select **FP32** only if you encounter numerical precision issues with FP16 or if deploying to older hardware with limited FP16 support. The exporter automatically falls back to FP32 if FP16 builder flags are unavailable.

### Where does RF-DETR save the TensorRT engine file?

The engine is saved to the directory specified by the `output_dir` parameter, using the naming convention defined in [`src/rfdetr/export/_naming.py`](https://github.com/roboflow/rf-detr/blob/main/src/rfdetr/export/_naming.py). The filename combines your specified `engine_name` with the precision suffix, producing files like `rfdetr_small-fp16.engine` or `rfdetr_small-fp32.engine` in the designated output directory.