How to Enable and Optimize Low VRAM Mode for TRELLIS.2 Inference

TRELLIS.2 provides a built-in low-VRAM mode that keeps model weights on CPU and streams only active layers to GPU, reducing peak memory usage below 6 GB with modest runtime overhead.

The low VRAM mode for TRELLIS.2 inference is enabled by default in Microsoft's open-source 3D generation pipeline. This mode implements on-demand GPU loading, where heavy sub-models remain on CPU and transfer to GPU only during forward passes. Understanding this mechanism helps you optimize for memory-constrained hardware or disable it for maximum speed on high-end GPUs.

How Low VRAM Mode Works in TRELLIS.2

The low VRAM architecture centers on the Trellis2ImageTo3DPipeline class in trellis2/pipelines/trellis2_image_to_3d.py. Rather than loading all model weights onto GPU at initialization, the pipeline selectively moves components based on the low_vram flag.

Pipeline Constructor and Device Management

The low_vram parameter is stored during initialization at lines 48-59:

def __init__(self, ..., low_vram=True, ...):
    ...
    self.low_vram = low_vram  # Default: True

The to(device) method at lines 99-105 demonstrates the core logic:

def to(self, device):
    if not self.low_vram:
        super().to(device)
        self.image_cond_model.to(device)
        if self.rembg_model is not None:
            self.rembg_model.to(device)
    # When low_vram=True, heavy sub-models stay on CPU

    return self

On-Demand GPU Loading Pattern

Every sub-model follows a consistent temporary-transfer pattern. The background removal model in preprocess_image (lines 44-49):

if self.low_vram:
    self.rembg_model.to(self.device)
output = self.rembg_model(input)
if self.low_vram:
    self.rembg_model.cpu()

The image conditioning transformer in get_cond (lines 71-79) uses identical mechanics:

if self.low_vram:
    self.image_cond_model.to(self.device)
cond = self.image_cond_model(image)
if self.low_vram:
    self.image_cond_model.cpu()

Shape-Latent Decoder Handling

The shape decoder requires additional state management at lines 321-326:

if self.low_vram:
    self.models['shape_slat_decoder'].to(self.device)
    self.models['shape_slat_decoder'].low_vram = True

# ... decode operations ...

if self.low_vram:
    self.models['shape_slat_decoder'].cpu()
    self.models['shape_slat_decoder'].low_vram = False

All heavy components—including the sparse-structure flow, shape-latent flow, texture-latent flow, and their decoders—follow this CPU-GPU streaming pattern.

Optimization Strategies for Low VRAM Inference

Essential Environment Configuration

Set PyTorch's CUDA allocator before importing torch:

import os
os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True"

This prevents large pre-allocation spikes that can cause OOM errors even when sufficient memory exists.

Runtime Memory Reduction Techniques

• Use torch.inference_mode() — Eliminates gradient buffers and autograd overhead
• Set batch size to 1 — The pipelines are designed for single-image generation; larger batches force simultaneous GPU residency
• Reduce target resolution — Lower from 1024³ to 512³ to shrink intermediate tensors
• Clear CUDA cache between runs — Optional for long-running processes:

import torch
torch.cuda.empty_cache()

Single GPU Assumption

The TRELLIS.2 codebase assumes single-GPU operation (torch.device('cuda')). Multi-GPU setups provide no additional memory savings because sub-models return to CPU after each use.

Code Examples: Three Common Configurations

Default Low VRAM Inference

import torch
import os
from PIL import Image
from trellis2.pipelines import Trellis2ImageTo3DPipeline

os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True"

pipeline = Trellis2ImageTo3DPipeline.from_pretrained(
    "microsoft/TRELLIS.2-4B",
    low_vram=True  # Explicit but redundant; True is default

)
pipeline.cuda()  # Only lightweight components move to GPU

with torch.inference_mode():
    mesh = pipeline.run(Image.open("input.png"))[0]

Optimized for Lower Memory with Reduced Resolution

pipeline = Trellis2ImageTo3DPipeline.from_pretrained(
    "microsoft/TRELLIS.2-4B",
    low_vram=True
)
pipeline.cuda()

with torch.inference_mode():
    mesh = pipeline.run(
        Image.open("input.png"),
        resolution=512  # Reduces tensor sizes throughout pipeline

    )[0]

High-Speed Mode: Disabling Low VRAM

For GPUs with 12+ GB VRAM, disable streaming to eliminate transfer overhead:

pipeline = Trellis2ImageTo3DPipeline.from_pretrained(
    "microsoft/TRELLIS.2-4B",
    low_vram=False
)
pipeline.to('cuda')  # All sub-models resident on GPU

with torch.inference_mode():
    mesh = pipeline.run(Image.open("input.png"))[0]

Key Source Files for Low VRAM Implementation

File	Purpose
trellis2/pipelines/trellis2_image_to_3d.py	Core low VRAM logic: constructor, to(), preprocessing, conditioning, decoder handling
trellis2/pipelines/trellis2_texturing.py	Identical pattern for texture-generation pipeline
trellis2/models/sc_vaes/sparse_unet_vae.py	Per-module low_vram attribute toggling
example.py / example_texturing.py	Reference scripts defaulting to low VRAM

Summary

• Low VRAM mode is default in Trellis2ImageTo3DPipeline via low_vram=True
• CPU-GPU streaming moves heavy models on-demand in preprocess_image, get_cond, and decoder operations
• Optimize with expandable_segments:True, torch.inference_mode(), resolution reduction, and batch size 1
• Disable for speed with low_vram=False on 12+ GB GPUs to keep all weights resident
• Monitor trellis2_image_to_3d.py lines 48-326 for implementation details

Frequently Asked Questions

What GPU memory does TRELLIS.2 low VRAM mode require?

Low VRAM mode typically keeps peak GPU usage below 6 GB according to the Microsoft TRELLIS.2 source code. Exact requirements depend on input resolution and which sub-models are active during each pipeline stage.

Why is my generation slower with low VRAM enabled?

The to(device) and cpu() transfers at lines 44-49, 71-79, and 321-326 introduce CPU-GPU communication overhead. Each sub-model loads to GPU for its forward pass then returns to CPU. This tradeoff reduces memory at the cost of transfer latency.

Can I use multiple GPUs to reduce memory further?

No. The codebase assumes single-GPU operation and moves sub-models back to CPU after use regardless of GPU count. Multi-GPU setups do not provide additional memory savings in the current implementation.

How do I verify low VRAM mode is active?

Check that pipeline.low_vram returns True and that pipeline.to('cuda') did not move heavy sub-models. You can also monitor GPU memory with nvidia-smi or torch.cuda.memory_summary() during generation—memory should spike during forward passes then drop between stages.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}