RF-DETR Training Hyperparameters: Complete Guide to TrainConfig and Model Optimization

RF-DETR training hyperparameters are centralized in the TrainConfig dataclass located in src/rfdetr/config.py, providing granular control over learning rates, automatic batch sizing, gradient accumulation, mixed precision, and Exponential Moving Average (EMA) behavior.

RF-DETR is a real-time transformer-based object detection model developed by Roboflow. Mastering the RF-DETR training hyperparameters through the TrainConfig interface is essential for optimizing GPU memory utilization, training convergence speed, and final detection accuracy on custom datasets.

Learning Rate and Optimizer Configuration

The learning rate schedule uses differentiated rates for the vision transformer backbone and detection heads. In src/rfdetr/config.py, TrainConfig defines these optimizer parameters:

• lr (float = 1e-4): Base learning rate for the detection head parameters.
• lr_encoder (float = 1.5e-4): Learning rate for the vision transformer encoder, typically set higher than the head to account for pre-trained feature extraction.
• lr_drop (int = 100): Epoch index at which the learning rate scheduler applies step decay.
• warmup_epochs (float = 0.0): Number of epochs for linear warmup before the main schedule begins.
• weight_decay (float = 1e-4): L2 regularization coefficient applied to all trainable parameters.

Layer-wise Learning Rate Decay

RF-DETR implements discriminative learning rates through decay factors that reduce learning rates in deeper network layers:

• lr_vit_layer_decay (float = 0.8): Decay multiplier applied progressively to deeper ViT encoder layers.
• lr_component_decay (float = 0.7): Decay factor for non-backbone components such as prediction heads.

Batch Size and Memory Management

RF-DETR features an intelligent auto-batch probing system that automatically maximizes GPU memory utilization. The relevant RF-DETR training hyperparameters include:

• batch_size (int | Literal["auto"] = 4): Micro-batch size per GPU. Setting "auto" triggers the auto-batch probe implemented in src/rfdetr/training/trainer.py.
• grad_accum_steps (int = 1): Number of gradient accumulation steps. The effective batch size equals batch_size × grad_accum_steps.
• auto_batch_target_effective (int = 16): Target global effective batch size. The probe automatically increases grad_accum_steps to reach this target while maximizing GPU memory.
• auto_batch_max_targets_per_image (int = 100): Upper bound on the number of targets per image considered during the memory probe.
• auto_batch_ema_headroom (float = 0.7): Safety scaling factor applied when EMA is enabled to account for the additional memory overhead of maintaining shadow weights.

Model Stability and Regularization

Control generalization and training stability through these regularization parameters:

• drop_path (float = 0.0): Stochastic depth probability for regularizing the transformer encoder during training.
• use_ema (bool = True): Enables maintenance of an Exponential Moving Average of model weights.
• ema_decay (float = 0.993): Decay factor for EMA weight updates.
• ema_tau (int = 100): Update interval for EMA synchronization, measured in optimizer steps.

Mixed Precision and Performance

Optimize training speed and memory with automatic mixed precision:

• amp_dtype ("auto" | "bf16" | "fp16"): Specifies the autocast datatype for mixed precision. The "auto" setting selects bf16 on Ampere+ GPUs and fp16 on older hardware.

Data Augmentation and Multi-Scale Training

Configure input preprocessing and augmentation strategies:

• multi_scale (bool = True): Enables multi-scale training with randomized image resolutions. Set False for static-shape training, which is required for TPU compatibility and reduces graph recompilations.
• expanded_scales (bool = True): Expands the range of randomized scales beyond the standard 2× range when multi-scale training is active.
• do_random_resize_via_padding (bool = False): Performs random resizing through padding rather than cropping when enabled.

Loss Function Weights

Adjust the contribution of individual loss components:

• cls_loss_coef (float = 1.0): Weight coefficient for the classification loss component.
• keypoint_*_loss_coef (float = 0): Individual weights for keypoint-specific losses including L1, findable, visible, and negative log-likelihood terms.

Training Duration and Monitoring

Manage training lifecycle and evaluation:

• epochs (int = 100): Total number of training epochs to run.
• checkpoint_interval (int = 10): Frequency of checkpoint saves, specified in epochs.
• early_stopping (bool = False): Enables early stopping based on validation metrics.
• early_stopping_patience (int = 10): Number of epochs to wait without improvement before triggering early stopping.
• eval_batch_size (int | None = None): Batch size for validation and inference loaders; defaults to the training batch size if unspecified.

Configuration Examples

Basic Training with Auto-Batch Detection

Configure training with automatic batch size detection to maximize GPU memory:

from rfdetr.config import TrainConfig
from rfdetr import RFDETR, get_model

train_cfg = TrainConfig(
    dataset_dir="my_dataset",
    output_dir="my_output",
    epochs=50,
    batch_size="auto",        # Auto-detect largest batch size

    use_ema=True,
    lr=2e-4,
    lr_drop=30,               # LR decay after 30 epochs

)

model_cfg = RFDETRSmallConfig()
model = get_model(model_cfg)

model.train(train_cfg)

Advanced Learning Rate Schedule with Mixed Precision

Implement layer-wise decay and force bfloat16 precision on modern hardware:

train_cfg = TrainConfig(
    dataset_dir="my_dataset",
    epochs=100,
    batch_size=4,
    grad_accum_steps=2,          # Effective batch = 8

    lr=1e-4,
    lr_encoder=2e-4,
    lr_vit_layer_decay=0.9,
    lr_component_decay=0.8,
    amp_dtype="bf16",            # Force bfloat16 on A100+

    use_ema=False,               # Disable EMA for faster iteration

    progress_bar="rich",
)

model.train(train_cfg)

Static-Shape Training for TPU Deployment

Disable multi-scale training to ensure fixed input dimensions:

train_cfg = TrainConfig(
    dataset_dir="my_dataset",
    epochs=80,
    batch_size=8,
    multi_scale=False,      # Fixed image shape for TPU

    use_ema=True,
    eval_batch_size=16,     # Larger batch for validation

)

model.train(train_cfg)

Key Implementation Files

Understanding these source files provides insight into how RF-DETR training hyperparameters are processed:

• src/rfdetr/config.py: Defines the TrainConfig dataclass and default values for all training parameters.
• src/rfdetr/training/trainer.py: Wraps PyTorch Lightning's Trainer and implements the auto-batch probing logic, EMA callbacks, and logging integration based on TrainConfig values.
• src/rfdetr/training/module_model.py: Implements the Lightning module that configures optimizers, learning rate schedules, and applies loss weighting according to the configuration.

Summary

• RF-DETR training hyperparameters are defined in the TrainConfig dataclass in src/rfdetr/config.py.
• Use batch_size="auto" combined with auto_batch_target_effective to automatically determine optimal micro-batch sizes and gradient accumulation steps.
• Set lr_encoder higher than lr to properly fine-tune the pre-trained vision transformer backbone.
• Enable multi_scale=True for improved scale invariance during training, or set multi_scale=False for TPU compatibility and deterministic memory usage.
• The auto_batch_ema_headroom parameter automatically adjusts batch calculations when EMA is enabled to prevent out-of-memory errors.
• Adjust cls_loss_coef and keypoint loss coefficients to balance multiple detection tasks.

Frequently Asked Questions

How does automatic batch sizing work in RF-DETR?

When batch_size="auto", the trainer implemented in src/rfdetr/training/trainer.py executes a memory probing routine that incrementally increases batch size until detecting memory limits. It then calculates the necessary grad_accum_steps to reach the auto_batch_target_effective global batch size while respecting auto_batch_ema_headroom and auto_batch_max_targets_per_image constraints.

What is the difference between lr and lr_encoder?

The lr parameter controls the learning rate for detection head parameters, while lr_encoder specifically targets the vision transformer backbone weights. According to the RF-DETR source code, lr_encoder defaults to 1.5e-4 compared to the base 1e-4 because pre-trained vision encoders typically require higher learning rates during fine-tuning to adapt pre-trained features to new domains.

Should I disable EMA during RF-DETR training?

Maintain use_ema=True for production training runs, as the Exponential Moving Average with ema_decay=0.993 and update interval ema_tau=100 significantly improves model stability and generalization. Disable EMA only during rapid prototyping or when facing severe memory constraints, since maintaining the EMA shadow weights requires approximately 50% additional memory capacity.

When should I use multi_scale=False?

Set multi_scale=False when training on TPUs to minimize graph recompilations caused by variable input dimensions, or when deterministic memory consumption is required for debugging. Static shapes also improve data loading consistency, though they may reduce final accuracy compared to multi-scale training which enhances the model's scale invariance.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}