How TRELLIS.2 Coordinates Its Three-Stage Sampling Pipeline: From Image to 3D Mesh

TRELLIS.2 generates complete 3D assets from a single image through a carefully orchestrated three-stage sampling process where each stage runs a dedicated flow-matching model and sampler, with outputs chained sequentially to produce final geometry and texture.

The three-stage sampling pipeline in microsoft/TRELLIS.2 transforms 2D images into high-fidelity 3D meshes through progressive refinement. This article examines how the pipeline components coordinate across sparse structure generation, shape latent diffusion, and texture synthesis—demonstrating the actual implementation in trellis2/pipelines/trellis2_image_to_3d.py and supporting modules.

The Three Stages of TRELLIS.2 Sampling

Stage	Output	Model	Sampler	Key Method
1	Binary voxel grid (coarse shape)	sparse_structure_flow_model	sparse_structure_sampler (FlowEulerSampler)	sample_sparse_structure()
2	Structured geometry latent (SparseTensor)	shape_slat_flow_model_512 or shape_slat_flow_model_1024	shape_slat_sampler (FlowEulerSampler or FlowEulerCfgSampler)	sample_shape_slat() / sample_shape_slat_cascade()
3	Texture latent conditioned on geometry	tex_slat_flow_model_512 or tex_slat_flow_model_1024	tex_slat_sampler (FlowEulerSampler or FlowEulerGuidanceIntervalSampler)	sample_tex_slat()

Each stage feeds its output forward as conditioning for the next, creating a coordinated sampling pipeline that builds complexity progressively.

Stage 1: Sparse Structure Sampling

The sparse structure stage establishes the coarse 3D occupancy of the object. In trellis2_image_to_3d.py (lines 64-86), the sample_sparse_structure() method:

1. Creates random noise in the shape of the target resolution
2. Invokes sparse_structure_sampler.sample() with image conditioning
3. Decodes the result through sparse_structure_decoder to produce occupied voxel coordinates

def sample_sparse_structure(self, cond, resolution=32, **kwargs):
    # Initialize random noise for sparse structure

    noise = torch.randn(batch_size, self.sparse_structure_flow_model.in_channels, 
                        resolution, resolution, resolution)
    
    # Run flow-matching sampler

    latents = self.sparse_structure_sampler.sample(
        model=self.sparse_structure_flow_model,
        noise=noise,
        cond=cond,
        **self.ss_sampler_params,
        **kwargs
    )
    
    # Decode to binary occupancy grid

    coords = self.sparse_structure_decoder.decode(latents)
    return coords

The sparse_structure_sampler is typically a FlowEulerSampler instantiated from trellis2/pipelines/samplers/flow_euler.py. This sampler implements Euler integration for flow-matching models, stepping from noise to data according to the learned velocity field.

Stage 2: Shape Latent Sampling

The shape latent stage generates detailed geometry at the coordinates discovered in Stage 1. The pipeline supports two modes:

• Single-resolution: sample_shape_slat() for 512³ or 1024³ outputs
• Cascade: sample_shape_slat_cascade() for progressive upsampling

Cascade Mode Implementation

The cascade mode (lines 86-95 in trellis2_image_to_3d.py) first generates a low-resolution latent, then upsamples:

def sample_shape_slat_cascade(self, lr_cond, cond, flow_model_lr, flow_model,
                              lr_resolution, resolution, coords, **kwargs):
    # Stage 2a: Low-resolution shape latent

    lr_noise = SparseTensor(
        feats=torch.randn(batch_size, flow_model_lr.out_channels, 
                         len(coords[lr_resolution])),
        coords=coords[lr_resolution]
    )
    lr_slat = self.shape_slat_sampler.sample(
        model=flow_model_lr,
        noise=lr_noise,
        cond=lr_cond,
        **self.shape_slat_sampler_params,
        **kwargs
    )
    
    # Prepare high-resolution coordinates and upsample conditioning

    hr_noise = SparseTensor(
        feats=torch.randn(batch_size, flow_model.out_channels,
                         len(coords[resolution])),
        coords=coords[resolution]
    )
    
    # Stage 2b: High-resolution refinement

    shape_slat = self.shape_slat_sampler.sample(
        model=flow_model,
        noise=hr_noise,
        cond=cond,
        # Concatenate upsampled LR latent as additional conditioning

        concat_cond=upsampled_lr_slat,
        **self.shape_slat_sampler_params,
        **kwargs
    )
    return shape_slat, lr_slat

The shape_slat_sampler uses FlowEulerCfgSampler when classifier-free guidance is enabled, mixing conditional and unconditional predictions via the ClassifierFreeGuidanceSamplerMixin.

Stage 3: Texture Latent Sampling

The texture stage synthesizes appearance information conditioned on the final geometry latent. In sample_tex_slat() (referenced in trellis2_image_to_3d.py), the pipeline:

1. Normalizes the shape latent from Stage 2
2. Concatenates it as concat_cond to the texture flow model
3. Runs the texture sampler with optional interval-based guidance

def sample_tex_slat(self, cond, flow_model, shape_slat, **kwargs):
    # Normalize and prepare geometry conditioning

    normalized_shape = normalize_sparse_tensor(shape_slat)
    
    # Initialize texture noise

    tex_noise = SparseTensor(
        feats=torch.randn(shape_slat.coords.shape[0], 
                         flow_model.out_channels),
        coords=shape_slat.coords
    )
    
    # Sample with geometry concatenated as conditioning

    tex_slat = self.tex_slat_sampler.sample(
        model=flow_model,
        noise=tex_noise,
        cond=cond,
        concat_cond=normalized_shape,  # Geometry guides texture

        **self.tex_slat_sampler_params,
        **kwargs
    )
    return tex_slat

The tex_slat_sampler may use FlowEulerGuidanceIntervalSampler (defined in flow_euler.py, lines 29-70) to apply guidance only during specific timestep intervals—a technique implemented via GuidanceIntervalSamplerMixin in guidance_interval_mixin.py (lines 9-13):

# From guidance_interval_mixin.py

def _inference_model(self, model, x_t, t, cond, **kwargs):
    if self.t_min <= t <= self.t_max:  # Inside guidance interval

        return super()._inference_model(model, x_t, t, cond, **kwargs)
    else:  # Outside interval: run unconditional

        return model(x_t, t, cond=None, **kwargs)

Pipeline Orchestration in run()

The run() method (starting at line 46 in trellis2_image_to_3d.py) sequences all three stages:

def run(self, image, pipeline_type="1024_cascade", low_vram=False, **kwargs):
    # Prepare conditioning once, reuse across stages

    cond_512 = self.get_cond([image], resolution=512)
    if pipeline_type == "1024_cascade":
        cond_1024 = self.get_cond([image], resolution=1024)
    
    # Stage 1: Sparse structure (resolution matches pipeline type)

    ss_res = 32 if "512" in pipeline_type else 64
    coords = self.sample_sparse_structure(cond_512, ss_res)
    
    # Stage 2: Shape latent (single or cascade)

    if pipeline_type == "1024_cascade":
        shape_slat, _ = self.sample_shape_slat_cascade(
            lr_cond=cond_512, cond=cond_1024,
            flow_model_lr=self.models['shape_slat_flow_model_512'],
            flow_model=self.models['shape_slat_flow_model_1024'],
            coords=coords
        )
    else:
        shape_slat = self.sample_shape_slat(cond_512, coords)
    
    # Stage 3: Texture latent

    tex_slat = self.sample_tex_slat(
        cond=cond_512 if "512" in pipeline_type else cond_1024,
        flow_model=self.models[f'tex_slat_flow_model_{resolution}'],
        shape_slat=shape_slat
    )
    
    # Decode to final mesh with PBR textures

    return self.decode_latent(shape_slat, tex_slat)

This sequential coordination ensures each stage receives appropriate conditioning: image features for stages 1-2, and geometry-aware conditioning for stage 3.

Conditioning and Sampler Configuration

Shared Conditioning Preparation

The get_cond() method extracts image features through image_cond_model, producing a conditioning dictionary passed to all samplers. For classifier-free guidance, a negative conditioning neg_cond is also prepared.

Runtime Sampler Overrides

Default sampler parameters load from the pipeline checkpoint JSON, but users can override per-stage behavior:

# Full pipeline with custom stage parameters

meshes = pipeline.run(
    image=Image.open("input.png"),
    pipeline_type="1024_cascade",
    ss_sampler_params={"num_steps": 50, "cfg_strength": 4.0},
    shape_slat_sampler_params={"num_steps": 25},
    tex_slat_sampler_params={"guidance_interval": (0.2, 0.8)}
)

Direct Sampler Instantiation

For advanced use, import and configure samplers directly from trellis2.pipelines.samplers:

from trellis2.pipelines.samplers.flow_euler import FlowEulerGuidanceIntervalSampler

# Custom sampler with narrow guidance window

custom_tex_sampler = FlowEulerGuidanceIntervalSampler(
    sigma_min=0.01,
    t_min=0.1, t_max=0.6  # Apply guidance only mid-diffusion

)

pipeline.tex_slat_sampler = custom_tex_sampler

Decoding and Output Formation

After sampling completes, decode_latent() (lines 56-86) transforms latents to renderable assets:

1. shape_slat_decoder converts geometry latent to mesh vertices and faces
2. tex_slat_decoder unpacks texture attributes to voxel-based PBR channels
3. Results bundle into MeshWithVoxel containing mesh and material attributes

Key Implementation Files

Path	Purpose
trellis2/pipelines/trellis2_image_to_3d.py	Main Trellis2ImageTo3DPipeline class; orchestrates all three sampling stages
trellis2/pipelines/samplers/flow_euler.py	FlowEulerSampler, FlowEulerCfgSampler, FlowEulerGuidanceIntervalSampler implementations
trellis2/pipelines/samplers/guidance_interval_mixin.py	GuidanceIntervalSamplerMixin for timestep-restricted guidance
trellis2/pipelines/samplers/classifier_free_guidance_mixin.py	Base CFG logic for conditional generation
trellis2/modules/sparse/basic.py	SparseTensor data structure for efficient 3D sparse convolutions
trellis2/models/sparse_structure_flow.py	Flow-matching model for Stage 1
trellis2/models/shape_slat_flow.py	Flow-matching models for Stage 2 (512³ and 1024³ variants)
trellis2/models/tex_slat_flow.py	Flow-matching models for Stage 3 texture synthesis

Summary

• TRELLIS.2's three-stage sampling pipeline progressively refines 3D generation: sparse structure → detailed geometry → texture, with each stage conditioning the next.

• Flow-matching samplers (FlowEulerSampler and variants in flow_euler.py) drive all stages, with mixins enabling classifier-free guidance and interval-based control.

• Coordinate chaining ensures geometric consistency: Stage 1 outputs voxel coordinates used to initialize Stage 2's SparseTensor, and Stage 2's normalized latent concatenates as conditioning for Stage 3.

• Pipeline flexibility supports single-resolution (512³) and cascade (512→1024³) modes through sample_shape_slat() and sample_shape_slat_cascade() respectively.

• Decoupled conditioning preparation via get_cond() allows efficient reuse of image features across all sampling stages.

Frequently Asked Questions

How does TRELLIS.2 ensure geometric consistency between sampling stages?

The pipeline passes occupied voxel coordinates from Stage 1 directly to Stage 2 via the coords parameter. Stage 2's SparseTensor is initialized at these exact locations, ensuring the detailed geometry latent occupies the same spatial region as the coarse binary structure. Stage 3 then inherits this coordinate system through the shape_slat tensor's coords attribute.

What is the difference between FlowEulerCfgSampler and FlowEulerGuidanceIntervalSampler?

FlowEulerCfgSampler (from flow_euler.py) applies classifier-free guidance at every timestep, scaling the conditional prediction against unconditional. FlowEulerGuidanceIntervalSampler inherits from it but adds timestep-restricted guidance via GuidanceIntervalSamplerMixin—only applying CFG when t falls within [t_min, t_max]. This reduces computational cost and can improve texture quality by allowing more natural stochasticity early and late in diffusion.

Can I run individual stages of the TRELLIS.2 pipeline independently?

Yes. The Trellis2ImageTo3DPipeline exposes separate methods: sample_sparse_structure(), sample_shape_slat(), sample_shape_slat_cascade(), and sample_tex_slat(). Each accepts pre-computed conditioning and can be called directly for debugging, interpolation experiments, or custom stage-wise modifications before final decoding.

What pipeline type should I use for highest quality versus fastest generation?

Use pipeline_type="1024_cascade" for highest quality—it runs all three stages with progressive upsampling (512→1024³) for detailed geometry. Use pipeline_type="512" for faster generation, which skips the cascade upsampler and uses smaller 512³ models for shape and texture. The cascade mode approximately doubles inference time but produces finer geometric detail.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}