# How to Handle NT/NN/TN/TT Transpose Conventions in DeepGEMM

> Learn to handle NT NN TN TT transpose conventions in DeepGEMM GEMM APIs. DeepGEMM requires explicit transposition for non-default layouts optimize performance.

- Repository: [DeepSeek/DeepGEMM](https://github.com/deepseek-ai/DeepGEMM)
- Tags: how-to-guide
- Published: 2026-04-19

---

**DeepGEMM requires explicit tensor transposition for non-default layouts, with NT being the native row-major/column-major format that requires no manual preparation, while NN, TN, and TT layouts require calling `.T.contiguous()` on inputs before invoking the corresponding kernel variants.**

The `deepseek-ai/DeepGEMM` repository provides just-in-time compiled GPU kernels for FP8 and FP4 matrix multiplication. Understanding how to handle the NT/NN/TN/TT transpose conventions is critical for correctly invoking these GEMM APIs, as the library does not perform implicit transpositions to maintain a lightweight kernels-only JIT path.

## Understanding the Four Layout Conventions

DeepGEMM follows standard BLAS naming conventions where two letters describe the storage format of matrices **A** and **B**:

- **First letter**: Storage of matrix **A** (row-major or column-major)
- **Second letter**: Storage of matrix **B** (row-major or column-major)

**N** denotes non-transposed (row-major for **A**, column-major for **B** in DeepGEMM's convention), while **T** denotes transposed (column-major for **A**, row-major for **B**).

## DeepGEMM's Default NT Layout

The **NT** layout is the native format for all DeepGEMM kernels. In this configuration:
- Matrix **A** is stored in row-major (non-transposed) order
- Matrix **B** is stored in column-major (transposed) order
- The kernel computes `D = C + A @ Bᵀ`

For NT operations, you can pass tensors directly without manual transposition:

```python
import torch
import deep_gemm

A = torch.randn(1024, 2048, device='cuda')
B = torch.randn(4096, 2048, device='cuda')  # column-major (transposed) storage

D = torch.empty_like(A @ B.T)

deep_gemm.fp8_gemm_nt(A, B, D)

```

## Handling NN, TN, and TT Layouts on SM 100

While SM 90 GPUs only support the NT layout, SM 100 (Blackwell) GPUs support all four transpose conventions. However, DeepGEMM does not perform implicit transpositions internally. You must explicitly transpose tensors before calling the corresponding API (`*_nn`, `*_tn`, `*_tt`).

### NN Layout (Both Operands Transposed)

For the **NN** layout, both **A** and **B** must be transposed from their native NT orientation:

```python
A_nn = A.T.contiguous()  # row-major → column-major

B_nn = B.T.contiguous()  # column-major → row-major

D_nn = torch.empty_like(A_nn @ B_nn.T)

deep_gemm.fp8_gemm_nn(A_nn, B_nn, D_nn)

```

### TN Layout (Only A Transposed)

For the **TN** layout, only matrix **A** requires transposition:

```python
A_tn = A.T.contiguous()
D_tn = torch.empty_like(A_tn @ B.T)

deep_gemm.fp8_gemm_tn(A_tn, B, D_tn)

```

### TT Layout (Both Operands + Scaling Factor Transposed)

The **TT** layout requires transposing both input matrices and the scaling-factor tensor. For FP4 operations, use the `transpose_packed_fp4` utility:

```python
from deep_gemm.utils.math import transpose_packed_fp4

A_tt = A.T.contiguous()
B_tt = B.T.contiguous()

# sf is a packed FP4 scaling-factor tensor from per_token_cast_to_fp4

sf_tt = transpose_packed_fp4(sf)

D_tt = torch.empty_like(A_tt @ B_tt.T)
deep_gemm.fp4_gemm_tt(A_tt, B_tt, D_tt, sf=sf_tt)

```

## Scaling Factor Transposition Requirements

The **scaling-factor tensor** (used for FP8/FP4 quantization) must always be provided in a **TMA-aligned and transposed** layout, regardless of the main GEMM layout. This constraint applies to both SM 90 and SM 100 architectures.

The `transpose_packed_fp4` function in [`deep_gemm/utils/math.py`](https://github.com/deepseek-ai/DeepGEMM/blob/main/deep_gemm/utils/math.py) handles the specific transposition logic for packed FP4 scaling factors when preparing TT layout operations.

## Key Implementation Files

| File | Purpose | Relevant Content |
|------|---------|------------------|
| [`deep_gemm/__init__.py`](https://github.com/deepseek-ai/DeepGEMM/blob/main/deep_gemm/__init__.py) | API surface | Exposes `fp8_gemm_nt`, `fp8_gemm_nn`, `fp8_gemm_tn`, `fp8_gemm_tt` and FP4 variants【28†L28-L53】 |
| [`tests/test_fp8_fp4.py`](https://github.com/deepseek-ai/DeepGEMM/blob/main/tests/test_fp8_fp4.py) | Usage examples | Demonstrates conditional transposition based on layout strings before kernel calls【25†L25-L27】【38†L38-L40】 |
| [`tests/generators.py`](https://github.com/deepseek-ai/DeepGEMM/blob/main/tests/generators.py) | Layout definitions | Defines `MajorTypeAB` enum mapping K-major (NT) vs MN-major (transposed) layouts【31†L31-L38】 |
| [`deep_gemm/utils/math.py`](https://github.com/deepseek-ai/DeepGEMM/blob/main/deep_gemm/utils/math.py) | Transposition utilities | Implements `transpose_packed_fp4` for scaling-factor transposition【104†L104-L118】 |
| [`README.md`](https://github.com/deepseek-ai/DeepGEMM/blob/main/README.md) | Architecture constraints | Documents NT-only support on SM 90 vs full layout support on SM 100【65†L65-L66】 |

## Summary

- **NT is the default**: DeepGEMM natively computes `D = C + A @ Bᵀ` with row-major **A** and column-major **B**; no manual transposition required.
- **SM 90 limitation**: Only NT kernels are available on Hopper architecture; other layouts require SM 100 (Blackwell).
- **Explicit transposition required**: For NN, TN, and TT layouts, you must call `.T.contiguous()` on the appropriate tensors before invoking `*_nn`, `*_tn`, or `*_tt` kernels.
- **Scaling factor constraints**: Quantization scaling factors must always be TMA-aligned and transposed; use `transpose_packed_fp4` for FP4 TT layouts.
- **Kernel naming convention**: The suffix (`_nt`, `_nn`, `_tn`, `_tt`) explicitly indicates the expected input layout, matching the BLAS convention.

## Frequently Asked Questions

### Does DeepGEMM automatically transpose matrices for NN or TT layouts?

No. DeepGEMM does not perform implicit transpositions. You must explicitly transpose tensors using `.T.contiguous()` before calling the corresponding kernel (e.g., `fp8_gemm_nn` or `fp8_gemm_tt`). This design keeps the JIT compilation path lightweight and avoids runtime overhead.

### Why do scaling factors need to be transposed separately from the main GEMM inputs?

The scaling-factor tensor follows its own layout constraint: it must always be **TMA-aligned and transposed** regardless of whether the main GEMM uses NT, NN, TN, or TT. This is a hardware requirement for the Tensor Memory Accelerator on NVIDIA GPUs. For FP4 operations in TT layouts, use the `transpose_packed_fp4` utility in [`deep_gemm/utils/math.py`](https://github.com/deepseek-ai/DeepGEMM/blob/main/deep_gemm/utils/math.py) to prepare the scaling factor correctly.

### Can I use NN, TN, or TT layouts on older GPUs like Hopper (SM 90)?

No. The NN, TN, and TT kernels are only available on SM 100 (Blackwell) GPUs. SM 90 (Hopper) only supports the NT layout. According to the README and the kernel dispatch logic in [`deep_gemm/__init__.py`](https://github.com/deepseek-ai/DeepGEMM/blob/main/deep_gemm/__init__.py), attempting to use non-NT layouts on SM 90 will result in compilation or runtime errors because the corresponding kernel variants are not compiled for that architecture.