# How to Configure DeepEP Low-Latency Mode QP (Queue Pair) for RDMA Optimization

> Optimize RDMA performance by configuring DeepEP low-latency mode QP. Learn how to set num_qps_per_rank for improved buffer creation and faster communication.

- Repository: [DeepSeek/DeepEP](https://github.com/deepseek-ai/DeepEP)
- Tags: how-to-guide
- Published: 2026-04-25

---

**To configure DeepEP low-latency mode Queue Pairs, set `num_qps_per_rank` equal to the number of local experts (`num_experts // world_size`) when creating the buffer with `low_latency_mode=True`.**

DeepEP is an open-source communication library designed by DeepSeek for Mixture-of-Experts (MoE) training and inference. When operating in low-latency mode, the library utilizes pure RDMA through an IBGDA (Infiniband Gather-DMA) transport, where each Queue Pair (QP) provides a dedicated RDMA channel to a local expert. Configuring the DeepEP low-latency mode QP correctly ensures minimal round-trip latency during dispatch and combine operations.

## Understanding the Queue Pair Requirement

DeepEP's low-latency architecture creates an IBGDA transport that maps each local expert to exactly one QP. This design eliminates contention by allowing the dispatch kernel to issue single RDMA writes per token without extra atomic operations.

### The One-to-One Mapping Rule

The most critical constraint when configuring DeepEP low-latency mode QP settings is the mandatory **1-to-1 mapping** between QPs and local experts. According to the repository's [`README.md`](https://github.com/deepseek-ai/DeepEP/blob/main/README.md) (lines 56-58), *"the QP number must be equal to the number of the local experts"*.

In [`deep_ep/buffer.py`](https://github.com/deepseek-ai/DeepEP/blob/main/deep_ep/buffer.py), the `Buffer` class initialization enforces this relationship. When `low_latency_mode=True` is specified, the constructor expects `num_qps_per_rank` to match the expert count per rank, calculated as:

```python
num_qps_per_rank = num_experts // world_size

```

### Why QP Count Matters for Performance

Each QP represents a dedicated RDMA channel to an expert's buffer. With the correct DeepEP low-latency mode QP configuration:

- **Direct addressing**: The dispatch kernel addresses experts directly via dedicated QPs
- **Reduced latency**: Eliminates fallback to generic RDMA paths that increase round-trip time
- **Ordering guarantees**: Maintains proper memory ordering without additional synchronization

If the QP count mismatches the local expert count, the library either raises an assertion error or silently falls back to higher-latency transport mechanisms, potentially violating performance expectations established in the low-latency design.

## Step-by-Step QP Configuration

Configuring the Queue Pairs requires three specific actions during buffer initialization.

### Calculate Local Experts

First, determine the number of experts residing on the current rank. DeepEP assumes uniform expert distribution across the world size:

```python
import torch.distributed as dist

world_size = dist.get_world_size()
local_experts = num_experts // world_size  # Must divide evenly

```

### Set low_latency_mode and num_qps_per_rank

In [`deep_ep/buffer.py`](https://github.com/deepseek-ai/DeepEP/blob/main/deep_ep/buffer.py) (lines 36-38), the `Buffer.__init__` method accepts the `low_latency_mode` boolean and `num_qps_per_rank` integer. You must set these explicitly:

```python
import deep_ep

buffer = deep_ep.Buffer(
    group=process_group,
    num_nvl_bytes=0,  # NVLink not used in pure low-latency mode

    num_rdma_bytes=deep_ep.Buffer.get_low_latency_rdma_size_hint(
        num_tokens, hidden, world_size, num_experts
    ),
    low_latency_mode=True,          # Enable IBGDA transport

    num_qps_per_rank=local_experts,  # ✅ Critical: QP count = expert count

    allow_nvlink_for_low_latency_mode=True,  # Set False for PCIe-only hardware

    explicitly_destroy=True
)

```

As shown in [`tests/test_low_latency.py`](https://github.com/deepseek-ai/DeepEP/blob/main/tests/test_low_latency.py) (lines 62-65), this pattern represents the standard initialization for low-latency workloads.

### Verify NVSHMEM Environment Variables

DeepEP translates the `num_qps_per_rank` parameter into the NVSHMEM environment variable `NVSHMEM_IBGDA_NUM_RC_PER_PE` (lines 107-110 in [`deep_ep/buffer.py`](https://github.com/deepseek-ai/DeepEP/blob/main/deep_ep/buffer.py)). Additionally, the code enforces a minimum QP depth via `NVSHMEM_QP_DEPTH` (default 1024) at lines 12-15 to prevent Work-Request slot exhaustion.

The library automatically sets `NVSHMEM_IB_ENABLE_IBGDA=1`, but you should verify your NVSHMEM build includes IBGDA support before runtime.

## Implementation Examples

### Manual Buffer Creation

For maximum control, instantiate the `Buffer` class directly as demonstrated in the test suite:

```python
import deep_ep
import torch.distributed as dist

# Setup distributed environment

group = dist.new_group()
rank, world_size = dist.get_rank(), dist.get_world_size()

# Model configuration

num_tokens = 128
hidden = 7168
num_experts = 64          # Must be divisible by world_size

num_topk = 8

# Calculate RDMA buffer requirements and QP count

rdma_bytes = deep_ep.Buffer.get_low_latency_rdma_size_hint(
    num_tokens, hidden, world_size, num_experts
)
num_qps = num_experts // world_size  # 8 QPs per rank if world_size=8

# Create the low-latency buffer

buffer = deep_ep.Buffer(
    group,
    num_nvl_bytes=0,
    num_rdma_bytes=rdma_bytes,
    low_latency_mode=True,
    num_qps_per_rank=num_qps,        # Maps 1 QP per local expert

    allow_nvlink_for_low_latency_mode=True,
    explicitly_destroy=True
)

```

### Using the get_buffer Helper

For inference servers and singleton patterns, use the `get_buffer` helper documented in the [`README.md`](https://github.com/deepseek-ai/DeepEP/blob/main/README.md):

```python
from deep_ep import get_buffer

# This helper automatically calculates local experts and QP requirements

buffer = get_buffer(
    group=process_group,
    num_max_dispatch_tokens_per_rank=128,
    hidden=7168,
    num_experts=64
)

```

The helper performs the QP-to-expert validation internally and constructs the buffer with the correct `num_qps_per_rank`.

### Runtime Dispatch and Combine Operations

Once configured, the DeepEP low-latency mode QP setup enables specific low-latency API methods:

```python

# Dispatch tokens to experts using the configured QPs

recv_hidden, recv_counts, handle, event, hook = buffer.low_latency_dispatch(
    hidden_states, 
    topk_idx,
    num_max_dispatch_tokens_per_rank=128,
    num_experts=64,
    async_finish=False,
    return_recv_hook=True
)

# Combine expert outputs

combined, event2, hook2 = buffer.low_latency_combine(
    recv_hidden, 
    topk_idx, 
    topk_weights, 
    handle
)

```

These methods leverage the IBGDA transport established during buffer creation, utilizing the dedicated QP channels for each expert.

## Troubleshooting and Validation

If you encounter initialization errors or performance degradation:

- **Assertion failures**: Verify that `num_experts` divides evenly by `world_size` and that `num_qps_per_rank` equals `num_experts // world_size`
- **NVLink compatibility**: Set `allow_nvlink_for_low_latency_mode=False` if running on PCIe-only hardware without NVLink bridges
- **QP depth warnings**: The library automatically sets `NVSHMEM_QP_DEPTH=1024` minimum; do not override this to lower values

## Summary

- **QP Requirement**: DeepEP low-latency mode requires exactly one QP per local expert (`num_experts // world_size`)
- **Configuration Location**: Set via `num_qps_per_rank` in `deep_ep.Buffer.__init__` or through the `get_buffer` helper
- **Environment Impact**: The parameter translates to `NVSHMEM_IBGDA_NUM_RC_PER_PE` in the underlying NVSHMEM transport
- **Validation**: Mismatched QP counts trigger assertions or fallback to higher-latency RDMA paths

## Frequently Asked Questions

### What happens if num_qps_per_rank does not match the local expert count?

The DeepEP runtime will raise an assertion error during buffer initialization, or in some configurations, silently fall back to a generic RDMA path that increases latency and may violate ordering guarantees required for correct MoE computation.

### Can I use NVLink instead of InfiniBand for low-latency mode?

Yes, but with restrictions. Set `allow_nvlink_for_low_latency_mode=True` to enable NVLink for low-latency operations. However, pure low-latency mode primarily targets InfiniBand RDMA via IBGDA; NVLink serves as an alternative transport when configured explicitly.

### Where is the QP configuration validated in the source code?

The validation occurs in [`deep_ep/buffer.py`](https://github.com/deepseek-ai/DeepEP/blob/main/deep_ep/buffer.py) (lines 107-110) where the library translates Python parameters to NVSHMEM environment variables, and in [`README.md`](https://github.com/deepseek-ai/DeepEP/blob/main/README.md) (lines 56-58) where the user-level helper performs sanity checks before buffer construction.

### Is there a minimum QP depth requirement for DeepEP?

Yes. According to lines 12-15 in [`deep_ep/buffer.py`](https://github.com/deepseek-ai/DeepEP/blob/main/deep_ep/buffer.py), the library enforces a minimum `NVSHMEM_QP_DEPTH` of 1024 to avoid Work-Request slot checks that would otherwise add latency to the critical path.