How to Install and Configure the DeepEP NVSHMEM Dependency

To install and configure the DeepEP NVSHMEM dependency, install NVIDIA NVSHMEM v3.3.9 or later, export the NVSHMEM_DIR environment variable pointing to the installation directory, and enable GPUDirect Async (IBGDA) for internode communication; if NVSHMEM_DIR is unset, DeepEP compiles with all internode and low-latency features disabled.

DeepEP, the expert parallelism communication library from the deepseek-ai/DeepEP repository, leverages NVIDIA NVSHMEM to enable high-performance GPU-to-GPU transfers across cluster nodes. Configuring this dependency correctly is essential for unlocking low-latency RDMA and internode communication features. This guide walks through the complete installation and configuration process based on the official source code.

Prerequisites

Before installing NVSHMEM, verify your hardware and software environment meets DeepEP's requirements.

Hardware Requirements:

• Intranode connectivity: GPUs within a single node must be connected via NVLink.
• Internode connectivity: GPUs across nodes require RDMA (InfiniBand) with GPUDirect Async (IBGDA) support, as documented in third-party/README.md【third-party/README.md line 11‑14】.

Software Requirements:

• NVSHMEM v3.3.9 or later is mandatory for compatibility with DeepEP's communication kernels【third-party/README.md line 18‑19】.

Installing NVSHMEM Binaries

NVSHMEM 3.3.9 can be obtained through multiple distribution channels. Choose the method that aligns with your system architecture (x86_64 or aarch64) and package management preferences:

• Tarballs: Download architecture-specific archives from NVIDIA【third-party/README.md line 24‑26】.
• Package managers: Install via RPM or DEB packages using the NVIDIA installer page【third-party/README.md line 26‑27】.
• Conda: Install from the conda-forge channel【third-party/README.md line 27】.
• pip: Install the wheel via pip install nvidia-nvshmem-cu12【third-party/README.md line 28】.

For tarball installation, extract the archive and note the installation path for the next step:

wget https://developer.download.nvidia.com/compute/nvshmem/redist/libnvshmem/linux-x86_64/libnvshmem-linux-x86_64-3.3.9_cuda12-archive.tar.xz
tar -xf libnvshmem-linux-x86_64-3.3.9_cuda12-archive.tar.xz

# Extracts to a directory like $HOME/nvshmem

Enabling IBGDA Support

IBGDA (IB GPUDirect Async) is required for low-latency internode communication. NVSHMEM supports two methods to enable this feature:

Method 1: Configure the NVIDIA driver

Add the following kernel module parameters to /etc/modprobe.d/nvidia.conf and reboot the system【third-party/README.md line 40‑45】:

sudo bash -c 'cat > /etc/modprobe.d/nvidia.conf <<EOF
options nvidia NVreg_EnableStreamMemOPs=1 NVreg_RegistryDwords="PeerMappingOverride=1;"
EOF'
sudo update-initramfs -u
sudo reboot

Method 2: Install GDRCopy

Download and install GDRCopy (available as deb/rpm or source), then load the gdrdrv kernel module following the upstream build instructions【third-party/README.md line 58‑62】.

Either method allows NVSHMEM to utilize IBGDA for RDMA operations.

Configuring Environment Variables

After installing NVSHMEM (particularly when using tarballs or manual builds), export the following environment variables so DeepEP can locate the library during compilation and runtime【third-party/README.md line 65‑70】:

export NVSHMEM_DIR=/path/to/your/nvshmem
export LD_LIBRARY_PATH="${NVSHMEM_DIR}/lib:$LD_LIBRARY_PATH"
export PATH="${NVSHMEM_DIR}/bin:$PATH"

Critical build-time dependency: The setup.py build script specifically checks for os.getenv('NVSHMEM_DIR')【setup.py line 20‑29】. When this variable is undefined, DeepEP prints a warning and automatically disables NVSHMEM-dependent code paths by injecting the -DDISABLE_NVSHMEM compiler flag【setup.py line 46‑49】.

Building DeepEP with NVSHMEM

With NVSHMEM installed and NVSHMEM_DIR set, build DeepEP from source:

NVSHMEM_DIR=/path/to/installed/nvshmem python setup.py install

The build system automatically adds the appropriate include paths and linker flags when NVSHMEM_DIR is detected. If the variable is missing, the build completes but excludes all internode and low-latency communication features.

Runtime Configuration

DeepEP exposes additional NVSHMEM tuning knobs via environment variables. These are processed at runtime and affect GPU resource mapping across nodes, as implemented in deep_ep/buffer.py【deep_ep/buffer.py line 108‑122】:

• NVSHMEM_DISABLE_P2P: Toggle NVLink peer-to-peer for low-latency mode.
• NVSHMEM_IB_ENABLE_IBGDA: Set to 1 to enable IBGDA for internode RDMA.
• NVSHMEM_IBGDA_NUM_RC_PER_PE: Configure the number of RC QPs per rank.
• NVSHMEM_QP_DEPTH: Adjust queue-pair depth (default: 1024).
• NVSHMEM_MAX_TEAMS: Set maximum NVSHMEM teams (default: 7).
• NVSHMEM_CUMEM_GRANULARITY: Define memory granularity (minimum 256 MiB).
• NVSHMEM_IB_SL: Control virtual lane assignment for IBGDA traffic.

Example runtime configuration:

export NVSHMEM_IB_ENABLE_IBGDA=1
export NVSHMEM_IBGDA_NUM_RC_PER_PE=4
export NVSHMEM_QP_DEPTH=2048
export NVSHMEM_CUMEM_GRANULARITY=268435456  # 256 MiB

python your_deep_ep_script.py

Summary

• NVSHMEM v3.3.9+ is required for DeepEP internode communication and low-latency GPU transfers.
• Set NVSHMEM_DIR before building; otherwise setup.py disables NVSHMEM features via -DDISABLE_NVSHMEM.
• IBGDA support is mandatory for internode RDMA and can be enabled via driver configuration or GDRCopy installation.
• Runtime behavior is controlled through variables like NVSHMEM_IB_ENABLE_IBGDA defined in deep_ep/buffer.py.

Frequently Asked Questions

What happens if I build DeepEP without setting NVSHMEM_DIR?

If NVSHMEM_DIR is not defined, DeepEP compiles successfully but automatically defines -DDISABLE_NVSHMEM, disabling all internode and low-latency communication features. The build process prints a warning indicating that NVSHMEM-dependent functionality is unavailable【setup.py line 29‑30】.

Is NVSHMEM required for single-node DeepEP deployments?

NVSHMEM is optional for single-node deployments that only use intranode NVLink communication. However, to enable internode RDMA communication across multiple nodes, NVSHMEM v3.3.9+ and IBGDA support are mandatory according to the hardware specifications in third-party/README.md.

How do I verify that IBGDA is properly enabled?

After configuring the NVIDIA driver or installing GDRCopy and loading the gdrdrv module, set NVSHMEM_IB_ENABLE_IBGDA=1 at runtime. DeepEP will utilize IBGDA for low-latency internode transfers if the underlying hardware (InfiniBand with GPUDirect Async) and kernel modules are correctly configured.

Can I use conda or pip instead of manual tarball extraction?

Yes. NVSHMEM is available via conda install -c conda-forge nvshmem or pip install nvidia-nvshmem-cu12【third-party/README.md line 27‑28】. If installing via these methods, you must still manually set NVSHMEM_DIR to the package installation location (e.g., within your conda environment or Python site-packages) before building DeepEP.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}