# SM90 vs SM100 Architecture Support in DeepGEMM: Hopper vs Ada Differences

> Discover the SM90 vs SM100 architecture support differences in DeepGEMM. Learn about Hopper's NT layout with FP32 scaling versus Ada's comprehensive layout support with packed UE8M0 scaling.

- Repository: [DeepSeek/DeepGEMM](https://github.com/deepseek-ai/DeepGEMM)
- Tags: deep-dive
- Published: 2026-04-19

---

**SM90 (Hopper) supports only NT layout with FP32 scaling factors, while SM100 (Ada) supports all four GEMM layouts (NT, TN, NN, TT) with packed UE8M0 scaling factors and requires CUDA 12.9+.**

DeepGEMM, the open-source FP8 GEMM library by DeepSeek, provides optimized matrix multiplication kernels for NVIDIA GPUs with distinct code paths for different compute architectures. Understanding the SM90 vs SM100 architecture support in DeepGEMM is essential for maximizing performance on Hopper (H100) versus Ada (RTX 40-series) hardware.

## Architectural Overview

DeepGEMM implements separate kernel families targeting specific NVIDIA GPU generations, with fundamental differences in memory layout support, scaling factor formats, and thread-level optimizations.

### SM90 (Hopper) Support

SM90 targets NVIDIA Hopper architecture GPUs with compute capability 9.0, such as the H100. According to the [`README.md`](https://github.com/deepseek-ai/DeepGEMM/blob/main/README.md), SM90 requires CUDA 12.3 or higher. The implementation resides primarily in `deep_gemm/include/deep_gemm/common/sm90_utils.cuh`, which defines SM90-specific utilities like `SM90_U32x2_LDSM_N` for low-level memory operations.

### SM100 (Ada) Support

SM100 targets NVIDIA Ada Lovelace architecture GPUs with compute capability 10.0, including RTX 40-series cards. The README specifies CUDA 12.9 or higher for SM100 support. The implementation uses `deep_gemm/include/deep_gemm/common/sm100_utils.cuh`, containing structures like `SM100_MMA_F16BF16_SS` for mixed-precision arithmetic.

## Key Differences in DeepGEMM Implementation

### CUDA Version Requirements

The two architectures have distinct minimum CUDA toolkit requirements. As documented in [`README.md`](https://github.com/deepseek-ai/DeepGEMM/blob/main/README.md) lines 29-36, SM90 requires CUDA 12.3 or higher, while SM100 requires CUDA 12.9 or higher. This reflects the different PTX instruction sets and driver features available on each generation.

### Memory Layout Constraints

The most significant functional difference lies in supported GEMM layouts. As stated in [`README.md`](https://github.com/deepseek-ai/DeepGEMM/blob/main/README.md) lines 65-66, the SM90 implementation supports only the **NT** layout (row-major A, column-major B). In contrast, the SM100 implementation supports all four classic GEMM layouts: **NT**, **TN**, **NN**, and **TT**.

This restriction in SM90 stems from Hopper's TMA (Tensor Memory Accelerator) constraints and the specific kernels implemented in `deep_gemm/include/deep_gemm/scheduler/gemm.cuh`, which notes SM90-specific scheduling limitations.

### Scaling Factor Formats

The two architectures require different scaling factor (SF) formats for FP8 quantization. According to [`README.md`](https://github.com/deepseek-ai/DeepGEMM/blob/main/README.md) lines 67-70:

- **SM90**: Requires **FP32** scaling factors
- **SM100**: Requires **packed UE8M0** format (4× UE8M0 packed into a single `torch.int`)

This difference affects how users must prepare input tensors. The SM100 path utilizes specialized packing utilities found in the SM100 kernel sources to handle the UE8M0 format efficiently.

### TMA and Multicast Optimization

The scheduler implementations reveal architectural differences in TMA multicast handling. In `deep_gemm/include/deep_gemm/scheduler/gemm.cuh` lines 108-112, comments indicate that SM90 can **dynamically disable** TMA multicast based on workload characteristics.

Conversely, SM100 uses a **fixed** multicast configuration as implemented in `deep_gemm/include/deep_gemm/common/sm100_utils.cuh`, with SM100-specific TMA utilities that handle the packed UE8M0 data paths.

### Kernel Architecture and Alignment

The alignment requirements differ significantly between the two:

- **SM90**: Uses an SF-aligned stride of **128** for float scaling factors, as defined in `deep_gemm/include/deep_gemm/scheduler/gemm.cuh` lines 35-36
- **SM100**: Uses an SF-aligned stride of **512** for packed UE8M0 formats

Additionally, SM90 kernels rely on **single-CTA** execution paths, while SM100 kernels can exploit **2-CTA** pipelines for higher throughput, as evidenced by the dual-CTA scheduling logic in the SM100 utils.

## Code Examples

The [`deep_gemm/__init__.py`](https://github.com/deepseek-ai/DeepGEMM/blob/main/deep_gemm/__init__.py) module provides a unified Python API that automatically dispatches to the appropriate kernel based on the runtime GPU's compute capability.

### SM90 (Hopper) Usage

When running on an H100 (SM90), DeepGEMM automatically selects kernels that require NT layout and FP32 scaling factors:

```python
import torch
import deep_gemm as dg

# SM90 requires NT layout: A row-major, B column-major

A = torch.randn(128, 256, dtype=torch.float16, device="cuda")
B = torch.randn(256, 128, dtype=torch.float16, device="cuda").t()

# Scaling factor must be FP32

sf = torch.randn(256, dtype=torch.float32, device="cuda")

# Automatically dispatches to SM90 kernel on H100

C = dg.fp8_gemm_nt(A, B, sf)

```

### SM100 (Ada) Usage

On RTX 40-series GPUs (SM100), the library supports all layouts and requires packed UE8M0 scaling factors:

```python
import torch
import deep_gemm as dg

# SM100 supports TN layout: A column-major, B row-major

A_tn = torch.randn(256, 128, dtype=torch.float16, device="cuda").t()
B_tn = torch.randn(128, 256, dtype=torch.float16, device="cuda")

# Scaling factor must be packed UE8M0 format

sf_float = torch.randn(128, dtype=torch.float32, device="cuda")
sf_packed = dg.pack_ue8m0(sf_float)

# Dispatches to SM100 kernel with TN layout support

C_tn = dg.fp8_gemm_tn(A_tn, B_tn, sf_packed)

```

## Summary

- **SM90 (Hopper)** targets compute capability 9.0 (H100) and requires CUDA 12.3+, supporting only **NT layout** with **FP32 scaling factors** and **128-byte alignment**.
- **SM100 (Ada)** targets compute capability 10.0 (RTX 40-series) and requires CUDA 12.9+, supporting **all four layouts** (NT, TN, NN, TT) with **packed UE8M0 scaling factors** and **512-byte alignment**.
- **Kernel architecture** differs in TMA multicast handling (dynamic vs fixed) and CTA execution (single-CTA for SM90, 2-CTA pipelines for SM100).
- **Implementation files** are located in `deep_gemm/include/deep_gemm/common/sm90_utils.cuh` and `deep_gemm/include/deep_gemm/common/sm100_utils.cuh`, with scheduling logic in `deep_gemm/include/deep_gemm/scheduler/gemm.cuh`.

## Frequently Asked Questions

### What CUDA version do I need for SM90 vs SM100 in DeepGEMM?

For SM90 (Hopper) GPUs like the H100, you need **CUDA 12.3 or higher**. For SM100 (Ada) GPUs like the RTX 4090, you need **CUDA 12.9 or higher**. These requirements are documented in the [`README.md`](https://github.com/deepseek-ai/DeepGEMM/blob/main/README.md) and reflect the different PTX instruction sets required for each architecture's TMA and MMA instructions.

### Why does SM90 only support NT layout while SM100 supports all layouts?

The SM90 implementation is optimized specifically for Hopper's TMA (Tensor Memory Accelerator) constraints and uses specialized kernels that only handle the NT (row-major A, column-major B) layout efficiently. In contrast, the SM100 implementation in `deep_gemm/include/deep_gemm/common/sm100_utils.cuh` contains more flexible MMA utilities that support all four standard GEMM layouts (NT, TN, NN, TT), leveraging the broader capabilities of the Ada architecture.

### What is the difference between FP32 and packed UE8M0 scaling factors?

SM90 requires **FP32 scaling factors** (32-bit floating point) for FP8 quantization, using a 128-byte alignment stride. SM100 requires **packed UE8M0 format**, which packs four 8-bit unsigned exponent values (UE8M0) into a single 32-bit integer, requiring 512-byte alignment. The packed format allows SM100 to process scaling factors more efficiently in its 2-CTA pipeline architecture, as implemented in the SM100 kernel utilities.