DeepEP Adaptive Routing vs Static Routing: Performance Differences and Configuration Guide

DeepEP adaptive routing introduces a small latency penalty (typically microseconds) to eliminate network congestion under heavy loads, while static routing provides the lowest possible per-message latency for light network traffic environments.

The deepseek-ai/DeepEP communication library, designed for Mixture-of-Experts (MoE) training and inference, supports two distinct network routing strategies on InfiniBand and RoCE fabrics. Understanding the performance difference between DeepEP adaptive routing and static routing is critical for maximizing throughput in large-scale GPU clusters.

How Routing Mode Affects Performance

The choice between routing modes creates a fundamental latency-versus-throughput trade-off in the network fabric. According to the DeepEP source documentation in README.md (lines 105-110), the implementation behaves differently depending on switch configuration:

• Static routing forces packets through fixed paths, minimizing per-hop decision time but risking path contention
• Adaptive routing dynamically balances traffic across available paths, adding minimal switch processing overhead but preventing congestion collapse

The performance impact manifests primarily at the transport layer rather than in DeepEP's CUDA kernels located in csrc/kernels/*, as the communication primitives in deep_ep/utils.py remain agnostic to the underlying path selection mechanism.

Static Routing: Minimal Latency for Light Loads

Static routing delivers the absolute lowest per-message latency by eliminating path selection overhead in the switch fabric. Packets follow predetermined routes from source to destination, allowing InfiniBand switches to forward traffic without computation.

This mode excels in environments with modest traffic where multiple flows rarely compete for the same physical links. However, as noted in the repository documentation, static routing may suffer from network congestion when many simultaneous flows target overlapping paths, causing throughput degradation that outweighs the latency benefits.

Configure static routing by restricting traffic to a single virtual lane:

import os

# Enable static routing via single virtual lane assignment

os.environ["NVSHMEM_IB_SL"] = "0"

Adaptive Routing: Congestion Elimination for Heavy Loads

Adaptive routing actively monitors link utilization and distributes packets across multiple available paths. While this adds a small processing delay (switches require additional time to select optimal routes), it completely eliminates network congestion caused by routing conflicts according to the README.md documentation (lines 105-108).

This mode is essential for large-scale training scenarios where hundreds or thousands of GPUs exchange activation data simultaneously. The microsecond-level latency penalty is negligible compared to the throughput gains achieved by avoiding hotspot links in the fabric.

Enable adaptive routing through virtual lane configuration:

import os

# Enable adaptive routing by allowing traffic distribution across lanes

os.environ["NVSHMEM_IB_SL"] = "1"

No additional code changes are required; DeepEP automatically utilizes the underlying adaptive routing capabilities when the InfiniBand fabric is configured accordingly.

Implementation and Testing

The routing choice does not alter the DeepEP API or buffer allocation logic found in deep_ep/utils.py. Whether using static or adaptive routing, the Buffer class and communication primitives function identically, with latency differences stemming entirely from the network fabric behavior.

The test suites in tests/test_intranode.py and tests/test_internode.py verify correct operation under both configurations, ensuring that routing mode changes only affect performance characteristics rather than correctness.

When benchmarking your specific workload, monitor both end-to-end latency and effective bandwidth. Static routing typically shows lower percentiles in latency distributions for small messages, while adaptive routing maintains stable bandwidth under heavy all-to-all communication patterns common in MoE training.

Summary

• Static routing provides the lowest possible per-message latency through fixed-path packet forwarding, ideal for latency-sensitive workloads with light network contention.
• Adaptive routing incurs a modest latency penalty (microseconds per message) to actively balance traffic across multiple paths, eliminating congestion in high-traffic GPU clusters.
• Configure the mode via the NVSHMEM_IB_SL environment variable (0 for static, 1 for adaptive) before initializing the DeepEP Buffer.
• The performance difference stems from InfiniBand switch behavior rather than DeepEP kernel code in csrc/kernels/*, requiring no API changes when switching modes.

Frequently Asked Questions

Does adaptive routing change DeepEP's CUDA kernel implementation?

No. The CUDA kernels in csrc/kernels/* and buffer management in deep_ep/utils.py operate identically regardless of routing mode. The performance difference between DeepEP adaptive routing and static routing occurs entirely at the network transport layer within the InfiniBand or RoCE switch fabric, not in the GPU code.

How do I switch between static and adaptive routing in DeepEP?

Set the NVSHMEM_IB_SL environment variable before importing DeepEP or initializing PyTorch distributed. Use os.environ["NVSHMEM_IB_SL"] = "0" for static routing (single virtual lane) or os.environ["NVSHMEM_IB_SL"] = "1" to enable adaptive path selection. The change requires no modifications to your Python application code.

When should I use static routing over adaptive routing?

Use static routing in environments with light network loads where traffic contention is unlikely, such as small-scale training runs or inference deployments with limited GPU counts. According to the DeepEP documentation in README.md (lines 109-110), static routing is recommended when you need the absolute lowest latency and the network fabric is not oversubscribed.

What latency penalty does adaptive routing add?

Adaptive routing adds a small, typically microsecond-level latency overhead per message because InfiniBand switches must compute optimal paths dynamically. However, this penalty is offset by significantly higher effective bandwidth under heavy loads, as adaptive routing prevents the congestion-induced throughput collapse that can occur with static routing when many GPU pairs communicate simultaneously.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}