# Does DeepEP Support Multi-Node NVLink (MNNVL)? Configuration and Usage Guide

> DeepEP enables Multi-Node NVLink (MNNVL) support. Learn how to configure and use MNNVL for enhanced distributed training performance via the allow_mnnvl flag or command-line option.

- Repository: [DeepSeek/DeepEP](https://github.com/deepseek-ai/DeepEP)
- Tags: how-to-guide
- Published: 2026-04-25

---

**Yes, DeepEP supports Multi-Node NVLink (MNNVL) when explicitly enabled via the `allow_mnnvl` flag in the `Buffer` class constructor or the `--allow-mnnvl` command-line option.**

The `deepseek-ai/DeepEP` library provides high-performance communication primitives for Mixture-of-Experts (MoE) training. While the library defaults to RDMA for inter-node traffic, it includes optional DeepEP Multi-Node NVLink support to leverage direct GPU-to-GPU links across server boundaries when the underlying NVSHMEM library and hardware infrastructure permit it.

## How MNNVL Support Works in DeepEP

DeepEP implements Multi-Node NVLink support through careful environment variable management during buffer initialization. The core logic determines whether the NVSHMEM runtime should attempt to detect cross-node NVLink connections.

### The `allow_mnnvl` Flag in [`buffer.py`](https://github.com/deepseek-ai/DeepEP/blob/main/buffer.py)

In [`deep_ep/buffer.py`](https://github.com/deepseek-ai/DeepEP/blob/main/deep_ep/buffer.py) (lines 24-40), the `Buffer` class constructor accepts an `allow_mnnvl` boolean parameter that defaults to `False`. When set to `True`, the library leaves the `NVSHMEM_DISABLE_MNNVL` environment variable unset, allowing the underlying NVSHMEM runtime to detect and utilize NVLink connections across nodes. When the flag remains `False`, DeepEP explicitly sets `NVSHMEM_DISABLE_MNNVL=1` before initializing NVSHMEM, effectively disabling MNNVL to prevent runtime errors on unsupported systems.

### Environment Variable Control

The implementation directly manipulates process environment variables before NVSHMEM initialization. This approach ensures that MNNVL activation requires explicit opt-in at the application level, preventing automatic attempts to use Multi-Node NVLink on clusters where the hardware or software stack does not support it.

## Enabling DeepEP Multi-Node NVLink

Activating MNNVL requires two conditions: explicit application-level configuration and a compatible NVSHMEM build. You can enable support either programmatically through the Python API or via command-line flags in the test suite.

### Python API Configuration

Pass `allow_mnnvl=True` when instantiating the `Buffer` class to permit NVSHMEM to use Multi-Node NVLink:

```python
import torch.distributed as dist
from deep_ep.buffer import Buffer

# Initialize your distributed process group

group = dist.group.WORLD

# Create buffer with MNNVL enabled

buffer = Buffer(
    group=group,
    num_nvl_bytes=64 * 1024 * 1024,      # IntrA-node NVLink buffer size

    num_rdma_bytes=128 * 1024 * 1024,    # Inter-node RDMA buffer size

    low_latency_mode=False,
    allow_mnnvl=True                   # Enable Multi-Node NVLink

)

```

### Command-Line Interface

The test harnesses expose the `--allow-mnnvl` flag for validation. As implemented in [`tests/test_intranode.py`](https://github.com/deepseek-ai/DeepEP/blob/main/tests/test_intranode.py) at line 306 and [`tests/test_low_latency.py`](https://github.com/deepseek-ai/DeepEP/blob/main/tests/test_low_latency.py) at line 322, you can run:

```bash
python tests/test_intranode.py --allow-mnnvl
python tests/test_low_latency.py --allow-mnvl

```

## System Requirements and Validation

For DeepEP to actually utilize MNNVL, your NVSHMEM installation must be compiled with Multi-Node NVLink support. Without this build configuration, setting `allow_mnnvl=True` has no effect on communication paths.

The library includes validation logic in [`deep_ep/utils.py`](https://github.com/deepseek-ai/DeepEP/blob/main/deep_ep/utils.py), where the `check_nvlink_connections` function inspects the GPU topology during buffer initialization. This ensures that the hardware is correctly detected before attempting MNNVL communication.

## Performance Implications

When enabled on compatible hardware, MNNVL allows DeepEP to use NVLink for inter-node communication in addition to standard RDMA pathways. This configuration can significantly improve bandwidth and reduce latency for collective operations spanning multiple nodes, particularly for large-scale MoE model training where expert parallelism requires frequent cross-node transfers.

## Summary

- DeepEP supports Multi-Node NVLink through the `allow_mnnvl` parameter in the `Buffer` class located in [`deep_ep/buffer.py`](https://github.com/deepseek-ai/DeepEP/blob/main/deep_ep/buffer.py).
- The default setting is `False`, which sets `NVSHMEM_DISABLE_MNNVL=1` to disable MNNVL and ensure compatibility.
- Enable it by setting `allow_mnnvl=True` in Python or using `--allow-mnnvl` when running the test scripts.
- Successful operation requires NVSHMEM built with MNNVL support and appropriate hardware infrastructure detected by `check_nvlink_connections` in [`deep_ep/utils.py`](https://github.com/deepseek-ai/DeepEP/blob/main/deep_ep/utils.py).

## Frequently Asked Questions

### What is Multi-Node NVLink (MNNVL) and why use it with DeepEP?

Multi-Node NVLink extends NVLink connectivity across server boundaries, allowing GPUs in different nodes to communicate directly at high bandwidth. Using MNNVL with DeepEP reduces latency and increases throughput for expert parallelism in distributed MoE training compared to pure InfiniBand or Ethernet RDMA solutions.

### Is MNNVL enabled by default in DeepEP?

No. To prevent compatibility issues on systems without MNNVL-capable hardware or NVSHMEM builds, DeepEP explicitly disables MNNVL by default via the `NVSHMEM_DISABLE_MNNVL` environment variable. You must set `allow_mnnvl=True` (or pass `--allow-mnnvl`) to permit the NVSHMEM runtime to detect and use cross-node NVLink connections.

### How can I verify my NVSHMEM installation supports MNNVL?

Check your NVSHMEM build configuration for MNNVL support flags, typically indicated during the configure or CMake stage. You can also inspect the [`deep_ep/utils.py`](https://github.com/deepseek-ai/DeepEP/blob/main/deep_ep/utils.py) file, which contains the `check_nvlink_connections` helper used during `Buffer` initialization to validate the NVLink topology across the allocated nodes.

### Can I use MNNVL with DeepEP's low-latency kernels?

Yes. The [`tests/test_low_latency.py`](https://github.com/deepseek-ai/DeepEP/blob/main/tests/test_low_latency.py) script accepts the `--allow-mnnvl` flag at line 322, confirming that low-latency communication kernels can operate over MNNVL when the feature is enabled. This allows high-frequency communication patterns to benefit from the lower latency of NVLink compared to traditional RDMA paths.