# How to Enable and Optimize Low VRAM Mode for TRELLIS.2 Inference

> Discover how to enable and optimize low VRAM mode for TRELLIS.2 inference. Reduce GPU memory usage below 6 GB and stream layers from CPU with minimal runtime impact.

- Repository: [Microsoft/TRELLIS.2](https://github.com/microsoft/TRELLIS.2)
- Tags: how-to-guide
- Published: 2026-08-04

---

**TRELLIS.2 provides a built-in low-VRAM mode that keeps model weights on CPU and streams only active layers to GPU, reducing peak memory usage below 6 GB with modest runtime overhead.**

The **low VRAM mode for TRELLIS.2 inference** is enabled by default in Microsoft's open-source 3D generation pipeline. This mode implements on-demand GPU loading, where heavy sub-models remain on CPU and transfer to GPU only during forward passes. Understanding this mechanism helps you optimize for memory-constrained hardware or disable it for maximum speed on high-end GPUs.

## How Low VRAM Mode Works in TRELLIS.2

The low VRAM architecture centers on the `Trellis2ImageTo3DPipeline` class in [`trellis2/pipelines/trellis2_image_to_3d.py`](https://github.com/microsoft/TRELLIS.2/blob/main/trellis2/pipelines/trellis2_image_to_3d.py). Rather than loading all model weights onto GPU at initialization, the pipeline selectively moves components based on the `low_vram` flag.

### Pipeline Constructor and Device Management

The `low_vram` parameter is stored during initialization at lines 48-59:

```python
def __init__(self, ..., low_vram=True, ...):
    ...
    self.low_vram = low_vram  # Default: True

```

The `to(device)` method at lines 99-105 demonstrates the core logic:

```python
def to(self, device):
    if not self.low_vram:
        super().to(device)
        self.image_cond_model.to(device)
        if self.rembg_model is not None:
            self.rembg_model.to(device)
    # When low_vram=True, heavy sub-models stay on CPU

    return self

```

### On-Demand GPU Loading Pattern

Every sub-model follows a consistent temporary-transfer pattern. The background removal model in `preprocess_image` (lines 44-49):

```python
if self.low_vram:
    self.rembg_model.to(self.device)
output = self.rembg_model(input)
if self.low_vram:
    self.rembg_model.cpu()

```

The image conditioning transformer in `get_cond` (lines 71-79) uses identical mechanics:

```python
if self.low_vram:
    self.image_cond_model.to(self.device)
cond = self.image_cond_model(image)
if self.low_vram:
    self.image_cond_model.cpu()

```

### Shape-Latent Decoder Handling

The shape decoder requires additional state management at lines 321-326:

```python
if self.low_vram:
    self.models['shape_slat_decoder'].to(self.device)
    self.models['shape_slat_decoder'].low_vram = True

# ... decode operations ...

if self.low_vram:
    self.models['shape_slat_decoder'].cpu()
    self.models['shape_slat_decoder'].low_vram = False

```

All heavy components—including the sparse-structure flow, shape-latent flow, texture-latent flow, and their decoders—follow this **CPU-GPU streaming pattern**.

## Optimization Strategies for Low VRAM Inference

### Essential Environment Configuration

Set PyTorch's CUDA allocator before importing torch:

```python
import os
os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True"

```

This prevents large pre-allocation spikes that can cause OOM errors even when sufficient memory exists.

### Runtime Memory Reduction Techniques

- **Use `torch.inference_mode()`** — Eliminates gradient buffers and autograd overhead
- **Set batch size to 1** — The pipelines are designed for single-image generation; larger batches force simultaneous GPU residency
- **Reduce target resolution** — Lower from 1024³ to 512³ to shrink intermediate tensors
- **Clear CUDA cache between runs** — Optional for long-running processes:

```python
import torch
torch.cuda.empty_cache()

```

### Single GPU Assumption

The TRELLIS.2 codebase assumes single-GPU operation (`torch.device('cuda')`). Multi-GPU setups provide no additional memory savings because sub-models return to CPU after each use.

## Code Examples: Three Common Configurations

### Default Low VRAM Inference

```python
import torch
import os
from PIL import Image
from trellis2.pipelines import Trellis2ImageTo3DPipeline

os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True"

pipeline = Trellis2ImageTo3DPipeline.from_pretrained(
    "microsoft/TRELLIS.2-4B",
    low_vram=True  # Explicit but redundant; True is default

)
pipeline.cuda()  # Only lightweight components move to GPU

with torch.inference_mode():
    mesh = pipeline.run(Image.open("input.png"))[0]

```

### Optimized for Lower Memory with Reduced Resolution

```python
pipeline = Trellis2ImageTo3DPipeline.from_pretrained(
    "microsoft/TRELLIS.2-4B",
    low_vram=True
)
pipeline.cuda()

with torch.inference_mode():
    mesh = pipeline.run(
        Image.open("input.png"),
        resolution=512  # Reduces tensor sizes throughout pipeline

    )[0]

```

### High-Speed Mode: Disabling Low VRAM

For GPUs with 12+ GB VRAM, disable streaming to eliminate transfer overhead:

```python
pipeline = Trellis2ImageTo3DPipeline.from_pretrained(
    "microsoft/TRELLIS.2-4B",
    low_vram=False
)
pipeline.to('cuda')  # All sub-models resident on GPU

with torch.inference_mode():
    mesh = pipeline.run(Image.open("input.png"))[0]

```

## Key Source Files for Low VRAM Implementation

| File | Purpose |
|------|---------|
| [`trellis2/pipelines/trellis2_image_to_3d.py`](https://github.com/microsoft/TRELLIS.2/blob/main/trellis2/pipelines/trellis2_image_to_3d.py) | Core low VRAM logic: constructor, `to()`, preprocessing, conditioning, decoder handling |
| [`trellis2/pipelines/trellis2_texturing.py`](https://github.com/microsoft/TRELLIS.2/blob/main/trellis2/pipelines/trellis2_texturing.py) | Identical pattern for texture-generation pipeline |
| [`trellis2/models/sc_vaes/sparse_unet_vae.py`](https://github.com/microsoft/TRELLIS.2/blob/main/trellis2/models/sc_vaes/sparse_unet_vae.py) | Per-module `low_vram` attribute toggling |
| [`example.py`](https://github.com/microsoft/TRELLIS.2/blob/main/example.py) / [`example_texturing.py`](https://github.com/microsoft/TRELLIS.2/blob/main/example_texturing.py) | Reference scripts defaulting to low VRAM |

## Summary

- **Low VRAM mode is default** in `Trellis2ImageTo3DPipeline` via `low_vram=True`
- **CPU-GPU streaming** moves heavy models on-demand in `preprocess_image`, `get_cond`, and decoder operations
- **Optimize with** `expandable_segments:True`, `torch.inference_mode()`, resolution reduction, and batch size 1
- **Disable for speed** with `low_vram=False` on 12+ GB GPUs to keep all weights resident
- **Monitor** [`trellis2_image_to_3d.py`](https://github.com/microsoft/TRELLIS.2/blob/main/trellis2_image_to_3d.py) lines 48-326 for implementation details

## Frequently Asked Questions

### What GPU memory does TRELLIS.2 low VRAM mode require?

Low VRAM mode typically keeps peak GPU usage below 6 GB according to the Microsoft TRELLIS.2 source code. Exact requirements depend on input resolution and which sub-models are active during each pipeline stage.

### Why is my generation slower with low VRAM enabled?

The `to(device)` and `cpu()` transfers at lines 44-49, 71-79, and 321-326 introduce CPU-GPU communication overhead. Each sub-model loads to GPU for its forward pass then returns to CPU. This tradeoff reduces memory at the cost of transfer latency.

### Can I use multiple GPUs to reduce memory further?

No. The codebase assumes single-GPU operation and moves sub-models back to CPU after use regardless of GPU count. Multi-GPU setups do not provide additional memory savings in the current implementation.

### How do I verify low VRAM mode is active?

Check that `pipeline.low_vram` returns `True` and that `pipeline.to('cuda')` did not move heavy sub-models. You can also monitor GPU memory with `nvidia-smi` or `torch.cuda.memory_summary()` during generation—memory should spike during forward passes then drop between stages.